Lightweight optical remote sensing image change detection method and system based on multi-scale perceptual learning, storage medium and electronic equipment

Abstract

The invention discloses a lightweight optical remote sensing image change detection method and system based on multi-scale perceptual learning, a storage medium and electronic equipment, and the method comprises the following steps: firstly, carrying out the preprocessing of a dual-time-phase optical image, marking a change region, and generating a label graph; then, the image and the label are cut into small samples, and the small samples serve as training data of an LCDNet model; according to the model, a lightweight encoder is adopted to extract a multi-scale feature map, a low-pass filtering down-sampling module reduces information loss, and a scale sensing block enhances the sensing ability of the model to different scale changes. And the multi-scale features are subjected to difference fusion through the spatio-temporal change perception module to generate a difference graph, efficient fusion decoding is realized through a learnable multi-scale decoder, and a final change graph is output. Model parameters are stored after training is completed, the preprocessed to-be-detected sample is input into the model, a detection change prediction map is output, accurate detection of a multi-scale change area is achieved while the parameters and the calculation amount are reduced, and the method is more suitable for actual deployment requirements.

Description

Technical Field

[0001] The present invention relates to the field of optical remote sensing image change detection, and in particular to a lightweight optical remote sensing image change detection method, system, storage medium and electronic device based on multi-scale perceptual learning. Background Art

[0002] Remote sensing image change detection aims to locate and segment surface changes in pairs of registered images acquired over the same area at different times. It is a key problem in remote sensing scene understanding. This technology, a key step in many real-world tasks, plays a vital role in global resource monitoring, land use change detection, disaster loss assessment, and urban management.

[0003] Early traditional remote sensing image change detection methods mainly rely on manually designed features to obtain change results, but these manual features usually lead to limited detection accuracy. In recent years, Convolutional Neural Networks (CNN) have been widely used in the field of image processing. [1] With its powerful feature learning ability, it has attracted widespread attention in the change detection task. Many CNN-based change detection methods [2] was proposed and achieved better performance than traditional methods. At the same time, the Transformer architecture [6] With its global modeling capabilities and flexible attention mechanism, it has gradually been introduced into the field of change detection, promoting the further development of this task in capturing long-distance dependencies and global context modeling.

[0004] The current research on change detection in remote sensing images focuses on improving detection accuracy. Some simpler models have low computational costs, but due to limited detection performance, it is difficult to generate accurate and complete change maps. Therefore, the mainstream trend is to adopt more complex network structures and training strategies to improve detection capabilities. However, the computational efficiency of these high-complexity models is low, which limits their promotion in large-scale remote sensing image processing and industrial applications with high real-time requirements. Some lightweight change detection (CD) networks

[10] It was proposed in an attempt to maintain high detection accuracy while reducing computational costs. However, most of the existing mainstream lightweight change detection methods rely on pre-trained classification networks (such as MobileNetV2). These networks were originally designed for natural image tasks and are difficult to fully adapt to the multi-scale target distribution and complex background differences commonly found in remote sensing images. Although such methods have achieved certain compression effects in terms of parameter quantity and computational overhead, they still suffer from problems such as parameter redundancy, insufficient structural generalization, and high inference costs in practical applications, making it difficult to strike a balance between performance and efficiency. Therefore, it is still necessary to design a more efficient lightweight model for remote sensing image change detection tasks that can significantly reduce model parameters and computational burden while maintaining or even improving detection accuracy, thereby better meeting deployment and application requirements in practical scenarios. Summary of the Invention

[0005] The purpose of the present invention is to provide a lightweight optical remote sensing image change detection method, system, storage medium and electronic device based on multi-scale perception learning. Through the synergy of lightweight design and multi-scale perception mechanism, efficient and accurate recognition of subtle changes between dual-phase optical remote sensing images can be achieved.

[0006] The technical solution adopted in the present invention is:

[0007] A lightweight optical remote sensing image change detection method based on multi-scale perceptual learning includes the following steps:

[0008] Step A, performing orthorectification, image registration, image stretching, and image numerical normalization preprocessing on the bi-temporal optical remote sensing image in sequence, thereby obtaining a bi-temporal optical remote sensing image with consistent data distribution;

[0009] Step B: labeling the updated part of the pre-processed bi-temporal optical remote sensing image obtained in step A to obtain a corresponding change label map;

[0010] Step C: Cut the change label map obtained in step B and the bi-temporal optical remote sensing image pre-processed in step A into the same size to obtain training samples;

[0011] Step D, create the network model LCDNet: The LCDNet model adopts a dual encoder-single decoder structure and adopts a mid-term fusion strategy (that is, the dual-temporal multi-scale features are fused layer by layer in the encoding stage). The whole model consists of three modules: a twin lightweight feature extraction encoder, which consists of a low-pass filter downsampling module (Blur Down) and multiple scale perception blocks (SABs). The number of SABs is set to 3, 7, 3, and 3 in different stages of feature extraction, achieving a balance between lightweight and multi-scale modeling; a lightweight spatiotemporal change perception module, which performs layer-by-layer difference fusion of dual-temporal multi-scale features; a multi-scale perception decoder, which is used to decode the difference features layer by layer and output the final change detection results;

[0012] Step E: Train the network model to obtain the optimal network model, which is used to detect the before and after phase remote sensing images to be detected, and output the detection results: input a pair of dual-temporal optical remote sensing images in the training sample into the network model created in step D, and sequentially undergo feature encoding, spatiotemporal change perception fusion, and multi-scale perception decoding to obtain the final predicted change map; then combine the binary cross entropy loss with the Dice loss to form a hybrid loss function to calculate the loss between the predicted change map of the dual-temporal image of the training sample and the corresponding label map, minimize the hybrid loss function through the back propagation algorithm, continuously update the model parameters, and save the optimal model.

[0013] The encoding and fusion process in step E specifically includes the following steps:

[0014] Step E1.1, first perform feature encoding through the reconstructed lightweight encoder to extract multi-scale feature maps X1, X2, X3, X4, Y1, Y2, Y3, Y4 respectively; where X i and Y i Representing the feature maps of phase 1 and phase 2 at the i-th scale, i = 1, 2, 3, 4, corresponding to feature representations at different scales from shallow to deep layers; the feature encoding operation in step E1.1 specifically includes the following steps: As shown in the figure, taking one stage as an example, first, a low-pass filtering downsampling (Blur Down) operation is performed on the input feature x. Compared with traditional downsampling methods, the present invention introduces a low-pass filtering operation during the downsampling process to effectively suppress aliasing and alleviate information loss, thereby preserving richer structural and edge features;

[0015] x m =MaxPool(x)

[0016] x c =Dwconv(x)

[0017] x d=Concat(Anti-aliased(x m ),Anti-aliased(x c ))

[0018] Among them, MaxPool(·) represents maximum pooling, Dwconv(·) represents depthwise convolution with stride=1 and kernel size of 3, and Anti-aliased(·) represents a set of low-pass filters with stride=2. Adding them after downsampling can effectively prevent high-frequency components from mixing into low-resolution features, reducing the generation of aliasing and artifacts. Subsequently, the downsampled features are evenly divided into two parts in the channel dimension. One part extracts multi-scale features through deep convolution layers with different convolution kernel sizes (k=1,3,5) set in parallel to enhance the model's perception of features of different scales; the other part is directly retained. Finally, the two parts of the features are spliced ​​and the channel shuffle operation is performed to promote information interaction between different groups, thereby further improving the feature expression ability and structural flexibility, and obtaining the final output x out .

[0019] x1,x2=Split(x d )

[0020] x2=Pconv(x2)

[0021] x2=∑D k wconv(x2)(k=1,3,5)

[0022] x2=Pconv(x2)

[0023] x out =Shuffle(Concat(x1,x2))

[0024] Where Pconv(·) represents 1×1 convolution, D k wconv(·) represents multi-scale convolution, and k represents the convolution kernel size.

[0025] Step E1.2: Input the bi-temporal feature maps of the same scale obtained in step E1.1 into the spatiotemporal change perception module for bi-temporal feature fusion. This module introduces attention mechanisms in both the channel and spatial dimensions, and combines the difference information of the bi-temporal features to guide the model to focus on potential change areas, thereby improving feature expression capabilities. Ultimately, four difference features D1, D2, D3, and D4 at different scales are obtained.

[0026] The bi-temporal fusion operation in step E1.2 specifically includes the following steps: First, the bi-temporal features (T1, T2) and the differential features (F d) perform global average pooling of space and channel respectively to capture the change information of different scales and dimensions of bi-temporal features. Here we take T1 as an example:

[0027] F d =(|T1-T2|)

[0028] S c1 =Concat(Avg(T1),Avg(F d ))

[0029] w c1 =Sigmoid(Con1D(S c1 ))

[0030] S c1 represents the aggregated spatial features, Avg(·) represents the global average pooling across spatial dimensions; F d The channel pooling results of T1 are transmitted in series to a one-dimensional convolution Con1D(·) to learn and determine the channel weights w related to the changes of the input dual-time features. c1 ; F d The introduction of explicitly guides the channel attention to focus more on the area where changes actually occur. A similar strategy is used in the spatial dimension to obtain the spatial weight w s1 The corresponding channel attention weights and spatial attention weights are aligned and added through the broadcast mechanism to form a unified comprehensive weight w1. The above operation is repeated on T2 to obtain w2. The original features are weighted with the weights to obtain a feature map with enhanced difference perception, providing a more robust and sensitive difference feature D for subsequent change detection. i support.

[0031] w1=w c1 +w s1

[0032] D i =w1×T1+w2×T2

[0033] The multi-scale perceptual decoding process in step E is to decode D1, D2, D3, and D4 in step E1.2, and perform decoding fusion layer by layer through a multi-scale perceptual decoder, specifically including the following steps:

[0034] Step E2.1: First, use 1×1 convolution to preliminarily fuse the difference features of adjacent layers;

[0035] F=Conv1(D i ,D i+1 )

[0036] Among them D i Indicates the difference features of different scales, i = 1, 2, 3;

[0037] Step E2.2, then, multi-scale information extraction is performed on the preliminary fusion features through convolution kernels of different scales (k = 1, 3, 5);

[0038] F i =D k wconv(F)

[0039] Among them D k wconv(·) represents multi-scale convolution, k represents the convolution kernel size;

[0040] Step E2.3, then, the multi-scale information is input into a learnable multi-scale perception module (MAM) with a weight sharing mechanism. It consists of three layers: first, global average pooling is performed to extract global context information, followed by feature mapping and nonlinear transformation through two fully connected (FC) layers. The module output is normalized by the Sigmoid function and used to dynamically adjust the weight distribution of feature streams of different scales. This module essentially constitutes a lightweight sub-neural network that can adaptively assign differentiated weights to features of different scales according to the input image. Finally, the weighted multi-scale feature map is element-by-element summed to generate a unified fusion feature map, which enhances the multi-scale adaptability and expression ability of feature representation;

[0041] F a =∑MAM(F i )

[0042] Step E2.4: Finally, the fused feature map is input into a 1×1 convolution block. After further feature extraction, it is superimposed with the preliminary fused feature map through a residual connection, and the fused decoded feature map is finally output.

[0043] F out =Conv1(F a )+F

[0044] Among them F out Represents the final fused decoding feature map, F a represents the fused multi-scale feature map, and F represents the preliminary fused feature map.

[0045] A lightweight optical remote sensing image change detection system based on multi-scale perception learning includes a preprocessing module for standardizing the input dual-temporal optical remote sensing images; a lightweight encoder for extracting multi-scale features of the dual-temporal images; a spatiotemporal change perception module for differentially fusing the multi-scale dual-temporal features output by two weight-shared lightweight encoders; a multi-scale decoder module for performing multi-scale fusion and decoding operations on adjacent scale difference features; the dual-temporal remote sensing images are preprocessed to ensure the alignment and consistency of the dual-temporal data and then input into the redesigned lightweight encoder to extract multi-scale features from the dual-temporal images layer by layer to form the basis for differential fusion; then, the dual-temporal features of the same scale are differentially fused scale by scale in the spatiotemporal change perception module to capture spatiotemporal change information and generate multi-scale difference features; finally, the multi-scale perception decoder module performs multi-scale fusion and decoding on the multi-scale difference features layer by layer during the decoding process and outputs the final change detection result map.

[0046] A computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the device where the computer-readable storage medium is located executes any one of the optical remote sensing image change detection methods based on change perception and semantic guidance.

[0047] An electronic device includes: a memory and a processor, wherein the memory stores a program that can be run on the processor, and when the processor executes the program, the lightweight optical remote sensing image change detection method based on multi-scale learning is implemented.

[0048] This paper, based on the application of change detection in optical remote sensing images, addresses the high parameter overhead and insufficient multi-scale feature extraction capabilities of existing change detection methods. It proposes a lightweight change detection method for optical remote sensing images based on multi-scale perceptual learning. This method balances efficiency and accuracy, effectively improving the ability to discern multi-scale target distributions and complex background differences in changing regions. Specifically, the present invention reconstructs a lightweight multi-scale encoder to fully extract multi-scale targets. It also integrates a spatiotemporal change perception mechanism based on bi-temporal features with a learnable multi-scale feature fusion decoding strategy. The former focuses on modeling differences in cross-temporal features. By introducing a channel and spatial attention mechanism and leveraging the complementary information between differential and bi-temporal features, it adaptively learns change-related channel and spatial weights to generate a feature map with enhanced perceptual perceptual performance. This provides more robust and sensitive feature support for subsequent change detection. The latter, by introducing a learnable multi-scale perceptual mechanism, achieves efficient fusion of features at different scales, thereby enhancing the model's ability to represent multi-scale change information. The collaborative design of these two methods not only improves the accuracy and robustness of change detection, but also enables efficient processing of complex high-resolution images within a lightweight design framework. Compared to traditional methods, this invention, while maintaining a lightweight model, addresses the common multi-scale target distribution and complex background differences in remote sensing images, fully exploiting the multi-scale information and change characteristics between dual-temporal images and effectively suppressing background interference. By introducing a multi-scale perception decoder module, combined with shared weights and efficient structural design, efficient fusion and precise expression of multi-scale features are achieved, significantly improving the model's modeling capabilities and expression integrity for changing areas. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0050] Figure 1 It is a schematic diagram of the process of the present invention.

[0051] Figure 2 This is the backbone network structure diagram of the LCDNet model described in the present invention.

[0052] Figure 3 This is a structural diagram of the lightweight encoder LWED described in the present invention.

[0053] Figure 4 This is a structural diagram of the lightweight spatiotemporal change perception module SCAM described in the present invention.

[0054] Figure 5 This is a structural diagram of the multi-scale perceptual decoder SAD described in the present invention. DETAILED DESCRIPTION

[0055] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without creative work are within the scope of protection of the present invention.

[0056] like Figure 1 As shown, the present invention includes the following steps:

[0057] Step A, performing orthorectification, image registration, image stretching, and image numerical normalization preprocessing on the bi-temporal optical remote sensing image in sequence, thereby obtaining a bi-temporal optical remote sensing image with consistent data distribution;

[0058] Step B: labeling the updated part of the pre-processed bi-temporal optical remote sensing image obtained in step A to obtain a corresponding change label map;

[0059] Step C: Cut the change label map obtained in step B and the bi-temporal optical remote sensing image pre-processed in step A into the same size to obtain training samples;

[0060] Step D, create the network model LCDNet: The LCDNet model adopts a dual encoder-single decoder structure and adopts a mid-term fusion strategy (that is, the dual-temporal multi-scale features are fused layer by layer in the encoding stage). The whole model consists of three modules: a twin lightweight feature extraction encoder, which consists of a low-pass filter downsampling module (Blur Down) and multiple scale perception blocks (SABs). The number of SABs is set to 3, 7, 3, and 3 in different stages of feature extraction, achieving a balance between lightweight and multi-scale modeling; a lightweight spatiotemporal change perception module, which performs layer-by-layer difference fusion of dual-temporal multi-scale features; a multi-scale perception decoder, which is used to decode the difference features layer by layer and output the final change detection results;

[0061] The lightweight encoder is used as a feature extractor such as Figure 3As shown in the figure, while mainstream MobileNetV2-based encoders excel in reducing computational overhead, they still suffer from deficiencies such as redundant computation, limited receptive field, and information loss due to downsampling. To address this, the present invention introduces a partial computation mechanism to reduce the computational burden of redundant channels, and combines it with multi-core deep convolution to enhance multi-scale feature modeling capabilities while reducing the number of parameters and computational complexity. Furthermore, an improved low-pass filtering downsampling module is designed to effectively alleviate the information loss issue. While maintaining lightweight, this approach achieves stronger feature expression capabilities and adaptability to complex scenarios.

[0062] The lightweight spatiotemporal change perception module is as follows Figure 4 As shown in the figure, traditional dual-temporal feature fusion methods often rely on a single feature map to generate attention weights, typically modeling only in the channel or spatial dimensions. This ignores the significant differences between the dual-temporal features, easily leading to insufficient response to subtle changes and sensitivity to background redundancy and texture interference, which affects detection performance. In contrast, the spatiotemporal change perception module explicitly introduces differential features as attention guides, combining channel and spatial dimensions for differential enhancement, effectively capturing fine-grained information in the changing region. Furthermore, the fusion strategy dynamically adjusts channel and spatial weights, improving the ability to focus on real changes and suppressing irrelevant information.

[0063] The multi-scale perceptual decoder is as follows Figure 5 As shown in the figure, traditional decoders usually use a fixed convolution kernel structure with a limited receptive field, making it difficult to effectively capture multi-scale change features. In addition, they lack a dynamic adjustment mechanism for semantic level information, which can easily lead to insufficient response to key change areas or feature confusion. To address the above problems, the present invention introduces a multi-core parallel deep convolution structure into the decoder to expand the scope of context perception and enhance the adaptability to multi-scale changes. At the same time, a learnable multi-scale perception mechanism is designed to adaptively adjust the weights of features at different scales according to image content, achieving more accurate and robust feature fusion and change area restoration, thereby improving overall change detection performance.

[0064] Step E: Train the network model to obtain the optimal network model, which is used to detect the before and after phase remote sensing images to be detected, and output the detection results: input a pair of dual-temporal optical remote sensing images in the training sample into the network model created in step D, and sequentially undergo feature encoding, spatiotemporal change guided fusion, and multi-scale perception decoding to obtain the final predicted change map; then combine the binary cross entropy loss with the Dice loss to form a hybrid loss function to calculate the loss between the predicted change map of the dual-temporal image of the training sample and the corresponding label map, minimize the hybrid loss function through the back propagation algorithm, continuously update the model parameters, and save the optimal model.

[0065] The encoding and fusion process in step E specifically includes the following steps:

[0066] Step E1.1, first perform feature encoding through the reconstructed lightweight encoder to extract multi-scale feature maps X1, X2, X3, X4, Y1, Y2, Y3, Y4 respectively; where X i and Y i Represent the feature maps of phase 1 and phase 2 at the i-th scale, i = 1, 2, 3, 4, corresponding to the feature representation of different scales from shallow to deep;

[0067] The feature encoding operation in step E1.1 specifically includes the following steps: As shown in the figure, taking one stage as an example, first, the input feature x is downsampled. Compared to traditional downsampling methods, the present invention introduces a low-pass filtering operation during the downsampling process, namely low-pass filtering downsampling (Blur Down), to effectively suppress aliasing and alleviate information loss, thereby preserving richer structural and edge features.

[0068] x m =MaxPool(x)

[0069] x c =Dwconv(x)

[0070] x d =Concat(Anti-aliased(x m ),Anti-aliased(x c ))

[0071] Among them, MaxPool(·) represents maximum pooling, Dwconv(·) represents depthwise convolution with stride=1 and kernel size of 3, and Anti-aliased(·) represents a set of low-pass filters with stride=2. Adding it after downsampling can effectively prevent high-frequency components from mixing into low-resolution features, reducing aliasing and artifacts. Subsequently, the downsampled features are evenly divided into two parts in the channel dimension. One part extracts multi-scale features through deep convolution layers with different convolution kernel sizes (k=1,3,5) set in parallel to enhance the model's perception of changes at different scales; the other part is directly retained. Finally, the two parts of the features are spliced ​​and the channel shuffle operation is performed to promote information interaction between different groups, thereby further improving the feature expression ability and structural flexibility, and obtaining the final output x out .

[0072] x1,x2=Split(x d )

[0073] x2=Pconv(x2)

[0074]

[0075] x2=Pconv(x2)

[0076] x out =Shuffle(Concat(x1,x2))

[0077] Where Pconv(·) represents 1x1 convolution, D k wconv(·) represents multi-scale convolution, and k represents the convolution kernel size.

[0078] Step E1.2: Input the bi-temporal feature maps of the same scale obtained in step E1.1 into the spatiotemporal change perception module for bi-temporal feature fusion. This module introduces attention mechanisms in both the channel and spatial dimensions, and combines the difference information of the bi-temporal features to guide the model to focus on potential change areas, thereby improving feature expression capabilities. Ultimately, four difference features D1, D2, D3, and D4 at different scales are obtained.

[0079] The bi-temporal fusion operation in step E1.2 specifically includes the following steps: First, the bi-temporal features (T1, T2) and the differential features (F d ) perform global average pooling of space and channel respectively to capture the change information of different scales and dimensions of bi-temporal features. Here we take T1 as an example:

[0080] F d =(|T1-T2|)

[0081] S c1 =Concat(Avg(T1),Avg(F d ))

[0082] w c1 =Sigmoid(Con1D(S c1 ))

[0083] S c1 represents the aggregated spatial features, Avg(·) represents the global average pooling across spatial dimensions; F d The channel pooling results of T1 are transmitted in series to a one-dimensional convolution Con1D(·) to learn and determine the channel weights w related to the changes of the input dual-time features. c1 ; F d The introduction of explicitly guides the channel attention to focus more on the area where changes actually occur. A similar strategy is used in the spatial dimension to obtain the spatial weight w s1The corresponding channel attention weights and spatial attention weights are aligned and added through the broadcast mechanism to form a unified comprehensive weight w1. The above operation is repeated on T2 to obtain w2. The original features are weighted with the weights to obtain a feature map with enhanced difference perception, providing a more robust and sensitive difference feature D for subsequent change detection. i support.

[0084] w1=w c1 +w s1

[0085] D i =w1×T1+w2×T2

[0086] The multi-scale perceptual decoding process in step E is to decode D1, D2, D3, and D4 in step E1.2, and perform decoding refinement layer by layer through the semantic guidance module, specifically including the following steps:

[0087] Step E2.1: First, use 1×1 convolution to preliminarily fuse the difference features of adjacent layers;

[0088] F=Conv1(D i ,D i+1 )

[0089] Among them D i Represents features of different scales, i=1, 2, 3.

[0090] Step E2.2, then, multi-scale information extraction is performed on the preliminary fusion features through convolution kernels of different scales (k = 1, 3, 5);

[0091] F i =D k wconv(F)

[0092] Among them D k wconv(·) represents multi-scale convolution, and k represents the convolution kernel size.

[0093] Step E2.3: Next, the multi-scale information is input into a learnable multi-scale perception module (MAM) with a weight sharing mechanism. It consists of three layers: first, global average pooling is performed to extract global context information, followed by feature mapping and nonlinear transformation through two fully connected (FC) layers. The module output is normalized by a sigmoid function and used to dynamically adjust the weight distribution of feature streams at different scales. This module essentially constitutes a lightweight sub-neural network that can adaptively assign differentiated weights to features at different scales based on the input image. Finally, the weighted multi-scale feature maps are element-wise summed to generate a unified fusion feature map, which enhances the multi-scale adaptability and expressiveness of the feature representation.

[0094] F a =∑MAM(F i )

[0095] Step E2.4: Finally, the fused feature map is input into the 1×1 convolution block. After further feature extraction, it is superimposed with the preliminary fused feature map through residual connection, and the fused decoding feature map is finally output.

[0096] F out =Conv1(F a )+F

[0097] Among them F out Represents the final fused decoding feature map, F a represents the fused multi-scale feature map, and F represents the preliminary fused feature map.

[0098] The use of the loss function, this application uses the cross entropy loss commonly used in binary classification tasks and Dice loss, which can alleviate background imbalance in samples Combined to form a mixing loss L = L bce +L dice To calculate the loss between the predicted change map of the bi-temporal image of the training sample obtained in step H and the corresponding label map, where y i,j Indicates the probability that the pixel (i, j) in the corresponding label image is a changed pixel, It indicates the probability that the pixel (i, j) in the predicted change map is a changed pixel, and n and m represent the width and height of the image pixel level respectively;

[0099] After the operation of steps AE and the completion of training, the present invention saves all parameter information of the trained model, thereby realizing the establishment of a change detection model. Subsequently, the remote sensing images of the before and after phases to be detected can be sequentially subjected to orthorectification, image registration, image stretching, and image numerical normalization preprocessing, and then cut into the same size to obtain samples to be detected; the samples to be tested are then input into the obtained change detection model, and a detection result graph is calculated and output.

[0100] A lightweight optical remote sensing image change detection system based on multi-scale perceptual learning includes a preprocessing module for normalizing the input bi-temporal optical remote sensing image; a lightweight encoder (LWED) for efficiently extracting multi-scale features of the bi-temporal image; a spatiotemporal change perception module (SCAM) for performing differential fusion of the multi-scale bi-temporal features output by two weight-sharing lightweight encoders; and a multi-scale perceptual decoder module (SAD) for performing multi-scale fusion and decoding operations on adjacent scale difference features (such as Figure 2); After preprocessing to ensure the alignment and consistency of the bi-temporal data, the bi-temporal remote sensing images are input into the redesigned lightweight encoder, which extracts multi-scale features layer by layer from the bi-temporal images to form the basis for difference fusion (such as Figure 3 ), and then, the bi-temporal features of the same scale are differentially fused scale by scale in the spatiotemporal change perception module to capture the spatiotemporal change information and generate multi-scale difference features (such as Figure 4 ); Finally, the multi-scale perception decoder module performs multi-scale fusion and decoding on the multi-scale difference features layer by layer during the decoding process, and outputs the final change detection result map (such as Figure 5 ).

[0101] This paper takes optical remote sensing image change detection as its application background. To address the problems of high parameter overhead and insufficient multi-scale feature extraction capabilities in existing change detection methods, the LCDNet model created by the present invention balances efficiency and accuracy, effectively improving the ability to recognize multi-scale target distribution and complex background differences in changing areas.

[0102] Specifically, the present invention reconstructs a lightweight multi-scale encoder to fully extract multi-scale targets. It also integrates the spatiotemporal variation perception mechanism of bi-temporal features with a learnable multi-scale feature fusion decoding strategy. The former focuses on modeling the differences between cross-temporal features. By introducing a channel and spatial attention mechanism and leveraging the complementary information between differential and bi-temporal features, it adaptively learns the channel weights and spatial weights associated with changes, generating a feature map with enhanced difference perception. This provides more robust and sensitive feature support for subsequent change detection. The latter, by introducing a learnable multi-scale perception mechanism, achieves efficient fusion of features at different scales, thereby enhancing the model's ability to express multi-scale variation information. The collaborative design of these two methods not only improves the accuracy and robustness of change detection but also enables efficient processing of complex high-resolution images within a lightweight design framework. Compared to traditional methods, the present invention, while maintaining a lightweight model, fully exploits the multi-scale information and variation characteristics between bi-temporal images, effectively suppressing background interference, addressing the common multi-scale target distribution and complex background differences in remote sensing imagery. By introducing a multi-scale perception decoder module and combining shared weights with efficient structural design, efficient fusion and precise expression of multi-scale features are achieved, thereby significantly improving the model's modeling ability and expression integrity of changing areas.

[0103] The embodiments of the present application provide a lightweight optical remote sensing image change detection method, system, storage medium, and electronic device based on multi-scale perception learning, which can implement the steps and processes of any of the above-mentioned lightweight optical remote sensing image change detection method embodiments based on multi-scale perception learning and achieve the same technical effects, and will not be repeated here one by one.

[0104] A computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the device containing the computer-readable storage medium executes the optical remote sensing image change detection method based on change perception and semantic guidance as described above. The computer program includes computer program code, which may be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium may include any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory, or other memory.

[0105] An electronic device includes: a memory and a processor, wherein the memory stores a program that can be run on the processor, and when the processor executes the program, the optical remote sensing image change detection method based on change perception and semantic guidance as described above is implemented.

[0106] If the modules / units integrated in the electronic device described in this application are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the present application can implement all or part of the processes in the above-mentioned embodiment methods by instructing the relevant hardware devices to complete them through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, it can implement the steps of the above-mentioned various method embodiments.

[0107] Furthermore, the computer-readable storage medium may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function, etc.; the data storage area may store data created according to the use of the blockchain node, etc.

[0108] The present invention is based on a change detection (CDD) dataset containing multiple change types.

[10] In order to verify the effectiveness of the proposed LCDNet, the following ten advanced remote sensing image change detection methods are selected for comparison with the proposed method, and they are briefly introduced.

[0109] FC-EF (Fully Convolutional-Early Fusion) [1]It is based on the U-Net architecture, in which bi-temporal images are cascaded into multi-band images for input, and skip connections are used to gradually transfer multi-scale features from the encoder to the decoder to recover spatial information. FC-Siam-conc (Fully Convolutional-Siamese-Concatenation) [1] As a variant of the FC-EF model, a Siamese encoder is used to extract features of bi-temporal images, and then the features of the same level from the encoder are connected to the decoder. FC-Siam-diff (Fully Convolutional-Siamese-Difference) [1] Unlike FC-Siam-conc, the skip connection of FC-Siam-diff is another kind of FC-EF model that transmits the absolute difference between the two temporal features. SNUNet-CD (Siamese NestedUNet-Change Detection) [2] The Siamese network is combined with the UNet++ network and the Ensemble Channel Attention Module (ECAM) is used to fuse the feature maps obtained from the backbone network at multiple semantic levels to suppress positioning errors and semantic gaps. [2] It is a ConvMixer-based network that only requires a small number of parameters to achieve good change detection performance. This method proposes a training method that learns detailed information from easy to difficult and an edge loss that focuses on the details of the network boundary. [3] The global features extracted by the ResNet18 network are input into the self-attention mechanism module and capture long-term spatiotemporal correlations to learn better representations. DASNet (Dual Attentive fully convolutional Siamese Networks) [5] Apply the attention mechanism to Siamese networks. BIT (Bitemporal Image Transformer) [9]The bi-temporal image is expressed as several semantics and the context is modeled in space and time based on compact semantics using a transformer encoder. The semantics are fed back to the pixel space and used to refine the original features through the transformer decoder. USSFC-Net (Ultralightweight Spatial–Spectral Feature Cooperation Network)

[10] A2Net (Progressive Feature Aggregation and Supervised Attention Network) is proposed. It has the dual advantages of lightweight structure and excellent accuracy. It can flexibly capture the multi-scale features of changing objects through multi-scale decoupled convolution and significantly reduce parameters and computational overhead.

[12] To address the problem of weak representation capabilities of mobile networks, this paper proposes to enhance temporal information extraction by fusing features from adjacent stages, and adopts progressive difference modeling and supervised attention guidance to achieve efficient aggregation of multi-level features and fine identification of changed areas.

[0110] Table I shows a comparative experiment conducted on the CDD dataset. Precision (P), recall (R), F1 score (F1Score, F1), and intersection over union (IoU) are used to quantitatively evaluate the performance of the methods involved. Parameters (Params), floating-point operations per second (FLOPs), and inference time are used to measure the computational effort and efficiency of the methods involved. Precision, recall, F1 score, and IoU metrics are calculated as follows:

[0111]

[0112] Here, true positives (TP) represent the number of correctly detected unchanged pixels, false positives (FP) represent the number of unpredicted unchanged pixels, and false negatives (FN) represent the number of unpredicted changed pixels. Precision represents the probability that all detected pixels have changed. Recall represents the probability that all changed pixels have been correctly detected. F1 is the harmonic mean of precision and recall, which balances conflicts by considering both precision and recall simultaneously. IoU is the area of ​​overlap between predicted changed pixels and changed pixels divided by their union area.

[0113] Table I Comparative experiments on the CDD dataset

[0114]

[0115] As can be seen from the data in the above table, the solution of the present invention improves 0.82% F1 and 1.49% IoU on the CDD dataset compared to other existing remote sensing image change detection methods. In addition, compared with other state-of-the-art lightweight change detection methods, the proposed LCDNet significantly reduces the number of parameters and floating-point operations (FLOPs) by about 50% while ensuring performance, further improving the computational efficiency and deployment friendliness of the model. The model proposed by the solution of the present invention achieves state-of-the-art change detection performance using only 0.84MB of parameters, 2.89GB of FLOPs, and an inference speed of 8.35ms. The solution of the present invention achieves the best performance on the CDD dataset and can identify changed areas with a lower number of parameters and faster speed.

[0116] In order to solve the problems of high parameter overhead and insufficient multi-scale feature extraction capabilities in existing change detection methods, the present invention constructs a lightweight network architecture based on multi-scale perception learning, called a lightweight change detection network (LCDNet), which takes into account both efficiency and accuracy, and effectively improves the ability to recognize multi-scale target distribution and complex background differences in changing areas. LCDNet greatly compresses parameters and computational complexity through a low-pass filtering downsampling module, a scale perception block, a spatiotemporal change perception module, and a multi-scale perception decoder. In order to realize the ability of the network to refine features and focus on changing areas, the present invention constructs a spatiotemporal change perception module based on channel attention and spatial attention. In order to achieve the effective fusion of feature maps of different scales, the present invention constructs a simple and effective multi-scale decoder.

[0117] The present invention proposes a lightweight remote sensing image change detection method based on multi-scale perception, which achieves faster inference speed while ensuring good change detection performance. By constructing an efficient encoding-decoding structure and integrating dual-phase difference modeling and multi-scale feature fusion mechanism, it improves detection accuracy and robustness while effectively coping with multi-scale target distribution and complex background interference in remote sensing images, and realizes efficient modeling and precise expression of changed areas.

[0118] The references in the invention are as follows:

[0119] [1]Daudt RC,Le Saux B,Boulch A.Fully convolutional siamese networks for change detection[C] / / 2018 25th IEEE International Conference on ImageProcessing(ICIP),2018:4063-4067.

[0120] [2]Fang S,Li K,Shao J,et al.SNUNet-CD:A densely connected Siamesenetwork for change detection of VHR images[J].IEEE Geoscience and RemoteSensing Letters,2021,19:1-5.

[0121] [3]Chen H,Pu F,Yang R,et al.RDP-Net:Region detail preserving networkfor change detection[J].IEEE Transactions on Geoscience and Remote Sensing,2022,60:1-10.

[0122] [4]Chen H,Shi Z.Aspatial-temporal attention-based method and a newdataset for remote sensing image change detection[J].Remote Sensing,2020,12(10):1662.

[0123] [5]Chen J,Yuan Z,Peng J,et al.DASNet:Dual attentive fullyconvolutional siamese networks for change detection in high-resolutionsatellite images[J].IEEE Journal of Selected Topics in Applied EarthObservations and Remote Sensing,2020,14:1194-1206.

[0124] [6]Vaswani A,Shazeer N,Parmar N,et al.Attention is all you need[C]. / / Proceedings of the 31st International Conference on Neural InformationProcessing Systems(NeurIPS).2017:6000–6010.

[0125] [7]Dosovitskiy A,Beyer L,Kolesnikov A,et al.An image is worth16x16words:Transformers for image recognition at scale[C].ProceedingsofInternational Conference on Learning Representations(ICLR),2021.

[0126] [8]Liu Z,Lin Y,Cao Y,et al.Swin transformer:Hierarchical visiontransformer using shifted windows[C]Proceedings of the IEEE / CVF InternationalConference on Computer Vision,2021:10012-10022.

[0127] [9]Chen H,Qi Z,Shi Z.Remote sensing image change detection withtransformers[J].IEEE Transactions on Geoscience and Remote Sensing,2021,60:1-14.

[0128]

[10] Bourdis N,Marraud D,Sahbi H.Constrained optical flow for aerialimage change detection[C]IEEE international geoscience and remote sensingsymposium,2011:4176-4179.

[0129]

[11] Lei T, Geng X, Ning H, et al. Ultralightweight spatial–spectralfeature cooperation network for change detection in remote sensing images [J]. IEEE Transactions on Geoscience and Remote Sensing, 2023, 61: 1-14.

[0130]

[12] Li Z, Tang C, Liu X, et al. Lightweight remote sensing change detection with progressive feature aggregation and supervised attention [J]. IEEE Transactions on Geoscience and Remote Sensing, 2023, 61: 1-12.

[0131] In the description of the present invention, it should be noted that, for directional words, such as the terms "center", "horizontal", "longitudinal", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise" and the like, indicating directions and positional relationships, are based on the directions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and cannot be understood as limiting the specific scope of protection of the present invention.

[0132] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0133] Note that the above are only preferred embodiments of the present invention and the principles of the technology used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments, and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention is described in detail through the above embodiments, the present invention is not limited to the specific embodiments described herein. Without departing from the concept of the present invention, it may also include many other effective embodiments, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A lightweight optical remote sensing image change detection method based on multi-scale perceptual learning, characterized by: It includes the following steps: Step A, performing orthorectification, image registration, image stretching, and image numerical normalization preprocessing on the bi-temporal optical remote sensing image in sequence, thereby obtaining a bi-temporal optical remote sensing image with consistent data distribution; Step B: labeling the updated part of the pre-processed bi-temporal optical remote sensing image obtained in step A to obtain a corresponding change label map; Step C: Cut the change label map obtained in step B and the bi-temporal optical remote sensing image pre-processed in step A into the same size to obtain training samples; Step D, create the network model LCDNet: The LCDNet model adopts a dual encoder-single decoder structure and adopts a mid-term fusion strategy, that is, fusing bi-temporal features scale by scale in the encoding stage; the whole model consists of three modules: a twin lightweight feature extraction encoder, a lightweight spatiotemporal change perception module, and a multi-scale perception decoder; among them, the twin lightweight feature extraction encoder consists of a low-pass filter downsampling module Blur Down and multiple scale perception blocks SAB. The number of SABs is set to 3, 7, 3, and 3 in different stages of feature extraction, achieving a balance between lightweight and multi-scale modeling; the lightweight spatiotemporal change perception module is used to perform scale-by-scale difference fusion of bi-temporal multi-scale features; the multi-scale perception decoder is used to decode the difference features layer by layer and output the final change detection results; Step E: Train the network model to obtain the optimal network model, which is used to detect the before and after phase remote sensing images to be detected, and output the detection results: input a pair of dual-temporal optical remote sensing images in the training sample into the network model created in step D, and sequentially undergo feature encoding, spatiotemporal change guided fusion, and multi-scale perception decoding to obtain the final predicted change map; then combine the binary cross entropy loss with the Dice loss to form a hybrid loss function to calculate the loss between the predicted change map of the dual-temporal image of the training sample and the corresponding label map, minimize the hybrid loss function through the back propagation algorithm, continuously update the model parameters, and save the optimal model.

2. The lightweight optical remote sensing image change detection method based on multi-scale perceptual learning according to claim 1 is characterized by: The feature encoding and spatiotemporal change perception fusion process in step E specifically includes the following steps: Step E1.1: First, perform feature encoding through a lightweight encoder to extract multi-scale feature maps X1, X2, X3, X4, Y1, Y2, Y3, Y4 respectively; where X i and Y i Represent the feature maps of phase 1 and phase 2 at the i-th scale, i = 1, 2, 3, 4, corresponding to the feature representation of different scales from shallow to deep; Step E1.2: Input the bi-temporal feature maps of the same scale obtained in step E1.1 into the spatiotemporal change perception module for bi-temporal feature fusion. This module introduces attention mechanisms in both the channel and spatial dimensions, and combines the difference information of the bi-temporal features to guide the model to focus on potential change areas, thereby improving feature expression capabilities and ultimately obtaining four different-scale difference features D1, D2, D3, and D4.

3. The lightweight optical remote sensing image change detection method based on multi-scale learning according to claim 1 is characterized by: The feature encoding operation in step E1.1 specifically includes the following steps: taking one stage as an example, first, low-pass filtering and downsampling Blur Down are performed on the input feature map x; X m =MaxPool(x) X c =Dwconv(x) X d =Concat(Anti-aliased(X m ),Anti-aliased(X c )) Among them, MaxPool(·) represents maximum pooling, Dwconv(·) represents depthwise convolution with stride=1 and kernel size of 3, and Anti-aliased(·) represents a set of low-pass filters with stride=2. Adding it before downsampling can effectively prevent high-frequency components from mixing into low-resolution features, reducing the generation of aliasing and artifacts. Subsequently, the downsampled features are evenly divided into two parts in the channel dimension. One part extracts multi-scale features through parallel deep convolution layers with different kernel sizes k to enhance the model's perception of changes at different scales; the other part is directly retained. Finally, the two parts of features are spliced ​​and shuffled through the channel to promote information interaction between different groups, thereby further improving feature expression ability and structural flexibility, and obtaining the final output x. out ; x1,x2=Split(X d ) x2=Pconv(x2) x2=Pconv(x2) x out =Shuffle(Concat(x1,x2)) Where Pconv(·) represents 1x1 convolution, D k wconv(·) represents multi-scale convolution, and k represents the convolution kernel size.

4. The lightweight optical remote sensing image change detection method based on multi-scale learning according to claim 2 is characterized in that: The bi-temporal fusion operation in step E1.2 specifically includes the following steps: First, the bi-temporal features T1, T2 and differential features F d Perform global average pooling of space and channel respectively to capture the change information of different scales and dimensions of bi-temporal features. Here we take T1 as an example: F d =(|T1-T2|) S c1 =Concat(Avg(T1),Avg(F d )) w c1 =Sigmoid(Con1D(S c1 )) S c1 represents the aggregated spatial features, Avg(·) represents the global average pooling across spatial dimensions; F d The channel pooling results of T1 are transmitted in series to a one-dimensional convolution Con1D(·) to learn and determine the channel weights w related to the changes of the input dual-time features. c1 ; F d The introduction of explicitly guides the channel attention to focus more on the area where changes actually occur. A similar strategy is used in the spatial dimension to obtain the spatial weight w s1 The corresponding channel attention weights and spatial attention weights are aligned and added through the broadcast mechanism to form a unified comprehensive weight w1. The above operation is repeated on T2 to obtain w2. The original features are weighted with the weights to obtain a feature map with enhanced difference perception, which provides a more robust and sensitive difference feature D for subsequent change detection. i support; w1=w c1 +in s1 D i =w1×T1+w2×T2。 5. The lightweight optical remote sensing image change detection method based on multi-scale perceptual learning according to claim 1 is characterized by: The multi-scale perceptual decoding process in step E is to decode D1, D2, D3, and D4 in step E1.3, and specifically includes the following steps: Step E2.1: First, use 1×1 convolution to preliminarily fuse the difference features of adjacent layers; F=Conv1(D i ,D i+1 ) Among them D i Indicates the difference features at different scales, i = 1, 2, 3; Step E2.2: Subsequently, multi-scale information extraction is performed on the preliminary fusion features using convolution kernels k of different scales; F i =D k wconv(F) Among them D k wconv(·) represents multi-scale convolution, k represents the convolution kernel size, k = 1, 3, 5; Step E2.3, then, the multi-scale information is input into a learnable multi-scale perception module (MAM) with a weight sharing mechanism; It consists of three layers: first, global average pooling is performed to extract global context information, followed by feature mapping and nonlinear transformation through two fully connected FC layers; The module output is normalized by the Sigmoid function and used to dynamically adjust the weight distribution of feature streams of different scales. This module essentially constitutes a lightweight sub-neural network that can adaptively assign differentiated weights to features of different scales according to the input image. Finally, the weighted multi-scale feature map is summed element by element to generate a unified multi-scale fusion feature map F. a , which enhances the multi-scale adaptability and expressiveness of feature representation; F a =∑MAM(F i ) Step E2.4: Finally, the multi-scale fused feature map is input into a 1×1 convolution block. After further feature extraction, it is superimposed with the preliminary fused feature map through a residual connection to effectively retain the original feature information, and finally the fused decoded feature map is output; F out =Conv1(F a )+F Among them F out Represents the final fused decoding feature map, F a represents the fused multi-scale feature map, and F represents the preliminary fused feature map.

6. A lightweight optical remote sensing image change detection system based on multi-scale perceptual learning, characterized by: include A preprocessing module for normalizing the input bi-temporal optical remote sensing image; A lightweight encoder module for efficient extraction of multi-scale features from bi-temporal images; The spatiotemporal change perception module is used to differentially fuse the multi-scale bi-temporal features output by two weight-sharing encoders; The multi-scale perception decoder module is used to fuse and decode the difference features of adjacent scales. After preprocessing to ensure the alignment and consistency of the bi-temporal remote sensing images, the bi-temporal remote sensing images are input into the encoder module. The encoder uses a reconstructed lightweight encoder to extract multi-scale features from the bi-temporal images layer by layer, forming the basis for difference fusion. Then, the bi-temporal features of the same scale are differentially fused scale by scale in the spatiotemporal change perception module to capture spatiotemporal change information and generate multi-scale difference features. Next, the multi-scale perception decoder module performs multi-scale fusion and decoding on the multi-scale difference features layer by layer, and outputs the final change detection result map.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the device where the computer-readable storage medium is located executes the lightweight optical remote sensing image change detection method based on multi-scale perception learning as described in any one of claims 1 to 5.

8. An electronic device, characterized in that: include: A memory and a processor, wherein the memory stores a program that can be run on the processor, and when the processor executes the program, it implements the lightweight optical remote sensing image change detection method based on multi-scale perception learning as described in any one of claims 1 to 5.

Images