Double-stage multi-scale change detection method for high-resolution remote sensing image

Abstract

The invention provides a double-stage multi-scale change detection method for a high-resolution remote sensing image, and the method comprises the steps: 1, carrying out the cascading of two stages of remote sensing images, and taking the cascaded images as input; extracting the features of the two-stage images through n coding levels in sequence; step 2, further extracting change features of different scales based on a jump connection structure of LDPPS; and step 3, outputting a detection result by a GLRAF module on the top of the decoder based on the GLRAF. The PGMCM effectively extracts multi-scale features of the image and enhances boundary information; and designing a hierarchical decreasing pyramid pooling structure LDPP S in jump connection, and further extracting multi-scale features of each hierarchical feature. In a decoding stage, a global-local relation perception feature fusion module GLRAF realizes effective fusion among different levels of features. And finally, an ambiguity weighted binary cross entropy loss function is given, so that the network pays more attention to difficult samples, and the detection effect on the boundary region is enhanced.

Description

Technical Field

[0001] The present invention relates to the field of remote sensing technology, and in particular to a dual-stage multi-scale change detection method for high-resolution remote sensing images. Background Art

[0002] In recent years, with the rapid development of deep learning technology, it has been introduced into the field of remote sensing change detection. Numerous deep learning-based change detection models and methods have been proposed. Thanks to the powerful feature representation capabilities of convolutional neural networks (CNNs), most existing deep learning change detection methods are based on CNNs. Based on their theoretical foundations, CNN-based deep learning change detection methods can be roughly divided into three categories: pure CNN change detection, attention-enhanced CNN change detection, and Transformer-integrated CNN change detection.

[0003] Pure CNN change detection is primarily achieved by optimizing classic semantic segmentation networks in computer vision. "Fully convolutional siamese networks for change detection" builds on fully convolutional neural networks (FCNs) and fuses low-level and high-level features of bi-temporal images via skip links to extract change information. "A Lightweight Dual-Branch Network for Building Change Detection in Remote Sensing Images Integrating Cross-Scale Coupling and Boundary Constraint" first utilizes a dual-branch CNN to extract image details and semantic information, respectively, and then implements change detection through cross-scale fusion and boundary constraints. "A deeply supervised image fusion network for change detection in high resolution bi-temporal remote sensing images" proposes a deeply supervised fusion change detection network for high-resolution remote sensing images based on the Siamese Unet network. "ACMFNet: Asymmetric Convolutional Feature Enhancement and Multiscale Fusion Network for Change Detection" improves the Unet network by using asymmetric convolution and multi-scale aggregation modules to implement change detection. "End-to-end change detection for high resolution satellite images using improved UNet++" improves the UNet++ network to detect change areas by using multilateral output fusion (MSOF) and deep supervision strategy.

[0004] Although many deep learning change detection methods have been proposed, remote sensing change detection remains an open problem due to numerous challenges. The main challenges in remote sensing change detection include: ① The scale and shape of objects changing in remote sensing imagery vary widely, which can lead to incomplete change detection, including holes and missed small objects; ② Deep learning models often require multiple downsampling steps, which can lead to loss of image detail and inaccurate extraction of change region boundaries. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide a two-stage multi-scale change detection method for high-resolution remote sensing images, which solves the above-mentioned problems of incomplete change detection, easy loss of image details, and inaccurate boundary extraction.

[0006] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0007] A two-stage multi-scale change detection method for high-resolution remote sensing images. The two-stage multi-scale change detection network TSMSNet adopts a U-shaped architecture and consists of three parts: an encoder based on PGMCM, a skip connection structure based on LDPPS, and a decoder based on GLRAF. The input part is two-phase remote sensing images, which are extracted by the multi-stage encoder based on PGMCM and connected with the skip connection structure based on LDPPS. The skip connection structure based on LDPPS is output to the multi-stage decoder based on GLRAF. The multi-stage encoder based on PGMCM and the multi-stage decoder based on GLRAF form a U-shape on both sides of the skip connection structure based on LDPPS. The change detection steps are as follows: Step 1, the encoder based on PGMCM concatenates the two-phase remote sensing images as the network input; then it passes through n encoding levels in sequence to extract the features of the two-phase images. The encoding levels are down-sampled to enter the next level, and the extraction results of each level are output to the skip connection structure based on LDPPS.

[0008] Step 2: The LDPPS-based skip connection structure is sequentially connected to multiple encoding levels of the PGMCM-based encoder. By considering the receptive field of each level feature, a hierarchical decreasing pyramid pooling structure LDPPS is formed to further extract the changing features of different scales. The hierarchical decreasing pyramid pooling structure LDPPS provides a multi-scale attention pyramid pooling module MAPPM for each level feature. The number of MAPPM layers decreases step by step as the receptive field of the level feature increases. MAPPM adaptively adjusts and enhances the features of different scales at each level through multi-scale attention and outputs the results to the GLRAF-based decoder.

[0009] Step 3. The GLRAF-based decoder contains n-1 global-local relationship perception feature fusion modules GLRAF and a fuzzy weighted enhanced prediction head. The n-1 GLRAF modules are directly connected to the n-1 level outputs of LDPPS from top to bottom. The output of the bottom layer of LDPPS, i.e. the nth layer, is connected to the GLRAF module of the adjacent upper layer through upsampling. The GLRAF modules are connected upward through upsampling. The top GLRAF module outputs the detection result through the fuzzy weighted enhanced prediction head.

[0010] The above n=5.

[0011] In the above-mentioned PGMCM-based encoder, among the five coding levels, for the first 1-3 coding levels, each level contains a traditional convolution block CBR and a probabilistic gated multi-scale convolutional modulation module PGMCM; the traditional convolution block CBR contains convolution Conv, batch normalization BN and Relu activation function in sequence; the probabilistic gated multi-scale convolutional modulation module PGMCM contains two sub-modules: multi-scale convolutional modulation module MCM and probabilistic gated module PGM; for the 4th-5th coding levels, each level contains a traditional convolution block CBR and a multi-scale convolutional modulation module MCM.

[0012] The probability gated multi-scale convolutional modulation module PGMCM in Step 1 above contains three groups of strip convolution optimized convolutional modulation modules. The specific details are as follows:

[0013]

[0014] Where X and Y represent the input and output of the MCM module, respectively. Conv and DConv represent normal convolution and depthwise convolution, respectively. The subscripts represent the convolution kernel size, k = 11-2×(i-1), q = 1+2×(i-1), and i = 1, 2, and 3. The PGM module adopts the idea of deep supervision. First, based on the output feature Y of the MCM, a prediction head PreH is used to generate a change probability prediction map P. Then, P is used to adaptively adjust and enhance the feature Y, and feature fusion is performed through residual connections. The specific process is as follows:

[0015]

[0016] Where Y and Z represent the input and output of the PGM module respectively, and PreH represents the prediction head, which consists of multiple convolutional layers.

[0017] The multi-scale attention pyramid pooling module for the first-level features in Step 2 above is MAPPM-1. MAPPM-1 consists of two stages: multi-scale feature extraction and feature enhancement. In the first stage, six scale features are generated through six groups of pooling operations: a spatial dimension global average pooling GAPs extracts global image features, and five local average poolings of different sizes and strides extract local features of different scales of the image. Then, channel interaction and spatial dimension restoration are performed on each scale feature through convolution and upsampling operations. Finally, the six scale features are fused through cascading. The details are as follows:

[0018]

[0019] Where F represents the input of MAPPM-1, f i represents the i-th level feature of the pyramid, f cRepresents cascade features, C represents cascade, Conv 1×1 represents 1×1 convolution, Up represents bilinear interpolation upsampling, GAPs represents global average pooling in spatial dimension, Avg (k,q) represents average pooling with size k×k and step size q;

[0020] In the feature enhancement stage, a multi-scale channel attention module is proposed to adaptively enhance features of different scales. A multi-scale channel attention module (MSCAM) is proposed. MSCAM consists of three branches: the first branch considers global context through global average pooling, the second branch considers local context through 3×3 pooling, and the third branch considers pixel-level context through 1×1 convolution. The specific details of MSCAM are as follows:

[0021]

[0022] Among them, GAPs represents the global average pooling of spatial dimensions, Avg (3,1) Indicates average pooling with a size of 3×3 and a step size of 1, Conv 1×1 represents 1×1 convolution, σ represents Sigmoid function; assuming that the output of MAPPM-1 is F ms , then F ms Finally, it is calculated by the following formula:

[0023] F ms =Conv 1×1 (C(f att ,Conv 1×1 (F)))+F;

[0024] Given that the receptive field of features at each level increases with the increase of the level, the proposed LDPPS uses MAPPM with decreasing pyramid levels: for the multi-scale pyramid pooling module MAPPM-2 of the second-level features, the maximum-scale local feature branch f5 in MAPPM-1 is removed; MAPPM-3, MAPPM-4 and MAPPM-5 of the 3rd to 5th level features can be obtained sequentially.

[0025] The global-local relationship perception feature fusion module GLRAF in Step 3 above consists of two stages: 1) modeling global relationships and 2) modeling local relationships. The specific processes of the two stages are as follows:

[0026] Step 3.1 Modeling global relationships; Calculate the global relationship matrix based on the channel attention of adjacent level features, using F s ∈R C×H×W and Represents shallow and deep features of adjacent levels;

[0027] Step 3.1.1. Perform channel interaction through 1×1 convolution and adjust F d Channel makes it have the same s The same number of channels, followed by spatial global average pooling GAPs and 1×1 convolution to generate F s and F d The channel attention vector att s and att d ;Step3.1.2, vector att s and att d Reorganize into r×k two-dimensional matrices, denoted as G s and G d , and calculate the global relationship GR matrix R by matrix multiplication g ;

[0028] Step 3.1.3, generate feature F s and F d The global enhancement weight M i (i=s or d), and for F s and F d Perform adaptive enhancement;

[0029] Step 3.2 Modeling local relationships; modeling enhanced features F by calculating the correlation between each pixel es and F ed Local relation LR;

[0030] Step 3.2.1, use bilinear interpolation to calculate F ed Upsample to make it consistent with F es With the same spatial resolution, F is generated by global maximum pooling GMPc and global average pooling GAPc in the channel dimension es and F ed The channel dimension is 2-dimensional matrix L s and L d ;

[0031] Step 3.2.2, calculate the local relationship matrix R l ;

[0032] Step 3.2.3, for feature F es and F ed Perform adaptive enhanced fusion.

[0033] The specific calculation process of the above Step 3.1 modeling global relationship is as follows:

[0034] Step 3.1.1. Perform channel interaction through 1×1 convolution and adjust F d Channel makes it have the same sThe same number of channels, followed by spatial global average pooling GAPs and 1×1 convolution to generate F s and F d The channel attention vector att s and att d :

[0035] att i =Conv 1×1 (GAPs(Conv 1×1 (F i )))i=sord;

[0036] Step 3.1.2, vector att s and att d Reorganize into r×k two-dimensional matrices, denoted as G s and G d , and calculate the global relationship GR matrix R by matrix multiplication g :

[0037]

[0038] Step 3.1.3, generate feature F s and F d The global enhancement weight M i (i=s or d), and for F s and f d Perform adaptive enhancement:

[0039]

[0040] Among them F ei (i=s or d) represents the enhanced features, Reshape represents the transformation matrix dimension, and σ represents the sigmoid function.

[0041] The specific calculation process of the above Step 3.2 modeling local relationship is as follows:

[0042] Step 3.2.1, use bilinear interpolation to calculate F ed Upsample to make it consistent with F es With the same spatial resolution, F is generated by global maximum pooling GMPc and global average pooling GAPc in the channel dimension es and F ed The channel dimension is 2-dimensional matrix L s and L d :

[0043] L i =C(GMPc(F ei ),GAPc(F ei))i=s ord;

[0044] Step 3.2.2, calculate the local relationship matrix R l :

[0045] R l =σ(L s1 iL d1 +L s2 iL d2 );

[0046] Where L sk and L dk Represents the matrix L s and L d The kth (k=1 or 2) channel dimension of , σ represents the sigmoid function;

[0047] Step 3.2.3, for feature F es and f ed Perform adaptive enhancement fusion:

[0048] F fusion =αiR l iF es +βiR l iF ed ;

[0049] Where α and β represent two learnable parameters.

[0050] The prediction head of the fuzzy weighted enhancement in Step 3 above regards the classification probability of the pixel as a fuzzy set, estimates the classification reliability of each pixel by calculating the fuzziness of the fuzzy set, and proposes a fuzzy weighted cross entropy loss function:

[0051] Assume p ic and p iu denotes the predicted probability of pixel i belonging to the changed class and the unchanged class (p ic +p iu =1), then the set {p ic ,p iu} is regarded as a fuzzy set, so its fuzziness can be calculated by the following formula to estimate the classification uncertainty Fu of pixel i i :

[0052]

[0053] Fu i The larger the value of is, the lower the classification reliability of pixel i is, and the more attention should be paid to pixel i; therefore, based on Fu i The loss weight w of pixel i is calculated by the following formula i :

[0054] w i =2*σ(Fu i -Fu);

[0055] Where Fu represents the average classification uncertainty of the pixel, σ represents the sigmoid function; the fuzzy weighted cross entropy loss function CE fuzz Defined by the following formula:

[0056]

[0057] Where log represents the natural logarithm operator, p ic represents the predicted probability that pixel i belongs to the change class, p iu represents the predicted probability that pixel i belongs to the unchanged class, y ic Is a Boolean variable. If pixel i belongs to the change class, then y ic =1, otherwise y ic =0;

[0058] To increase the diversity of the loss, the proposed ambiguity-weighted cross entropy loss function CE is used in the final decoding stage. fuzz , in the deep supervision of the encoding stage, the traditional cross entropy loss CE is adopted; the total loss is as follows:

[0059]

[0060] Among them CE l represents the traditional cross entropy loss of layer l in deep supervision.

[0061] The present invention provides a two-stage multi-scale change detection method for high-resolution remote sensing images. First, in the encoding stage, based on convolution modulation, strip convolution and gating mechanism, a probabilistic gated multi-scale convolution modulation module PGMCM is proposed to effectively extract the multi-scale features of the image and enhance the boundary information. Secondly, by considering the receptive field of features at each level, a hierarchical decreasing pyramid pooling structure LDPPS is designed in the jump connection to further extract the multi-scale features of features at each level. Then, in the decoding stage, a global-local relationship perception feature fusion module GLRAF is constructed to achieve effective fusion between features at different levels. Finally, a fuzzy weighted binary cross entropy loss function is given to make the network pay more attention to difficult samples, thereby enhancing the detection effect of boundary areas. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] The present invention will be further described below with reference to the accompanying drawings and examples:

[0063] Figure 1 This is a schematic diagram of the structure of the dual-stage multi-scale change detection network TSMSNet for high-resolution remote sensing images of the present invention;

[0064] Figure 2 Schematic diagram of the traditional convolutional modulation module (a) and the probabilistic gated multi-scale convolutional modulation module PGMCM (b);

[0065] Figure 3 This is a schematic diagram of the structure of the first-level multi-scale attention pyramid pooling module MAPPM-1 in the present invention;

[0066] Figure 4 Schematic diagram of the structure of the global-local relationship perception feature fusion module GLRAF in the present invention;

[0067] Figure 5 Schematic diagram of visualization results of different networks on the WHU dataset in an embodiment of the present invention;

[0068] Figure 6 Schematic diagram of visualization results of different networks on the GZ dataset in an embodiment of the present invention;

[0069] Figure 7 Schematic diagram of visualization results of different networks on the GVLM dataset in an embodiment of the present invention. DETAILED DESCRIPTION

[0070] The technical solution of the present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0071] The two-stage multi-scale change detection network TSMSNet adopts a U-shaped architecture, which consists of three parts: an encoder based on PGMCM, a skip connection structure based on LDPPS, and a decoder based on GLRAF. The input part is a two-phase remote sensing image, which is extracted by the multi-stage encoder based on PGMCM and connected with the skip connection structure based on LDPPS. The skip connection structure based on LDPPS outputs to the multi-stage decoder based on GLRAF. The multi-stage encoder based on PGMCM and the multi-stage decoder based on GLRAF form a U-shape on both sides of the skip connection structure based on LDPPS. The change detection steps are as follows:

[0072] Step 1: The encoder based on PGMCM concatenates two remote sensing images as network input. It then passes through n encoding levels in sequence to extract the features of the two images. The encoding levels are downsampled to the next level, and the extraction results of each level are output to the jump connection structure based on LDPPS.

[0073] Step 2: The LDPPS-based skip connection structure is sequentially connected to multiple encoding levels of the PGMCM-based encoder. By considering the receptive field of each level feature, a hierarchical decreasing pyramid pooling structure LDPPS is formed to further extract the changing features of different scales. The hierarchical decreasing pyramid pooling structure LDPPS provides a multi-scale attention pyramid pooling module MAPPM for each level feature. The number of MAPPM layers decreases step by step as the receptive field of the level feature increases. MAPPM adaptively adjusts and enhances the features of different scales at each level through multi-scale attention and outputs the results to the GLRAF-based decoder.

[0074] Step 3. The GLRAF-based decoder contains n-1 global-local relationship perception feature fusion modules GLRAF and a fuzzy weighted enhanced prediction head. The n-1 GLRAF modules are directly connected to the n-1 level outputs of LDPPS from top to bottom. The output of the bottom layer of LDPPS, i.e. the nth layer, is connected to the GLRAF module of the adjacent upper layer through upsampling. The GLRAF modules are connected upward through upsampling. The top GLRAF module outputs the detection result through the fuzzy weighted enhanced prediction head.

[0075] The above n=5.

[0076] In the above-mentioned PGMCM-based encoder, among the five coding levels, for the first 1-3 coding levels, each level contains a traditional convolution block CBR and a probabilistic gated multi-scale convolutional modulation module PGMCM; the traditional convolution block CBR contains convolution Conv, batch normalization BN and Relu activation function in sequence; the probabilistic gated multi-scale convolutional modulation module PGMCM contains two sub-modules: multi-scale convolutional modulation module MCM and probabilistic gated module PGM; for the 4th-5th coding levels, each level contains a traditional convolution block CBR and a multi-scale convolutional modulation module MCM.

[0077] The probability gated multi-scale convolutional modulation module PGMCM in Step 1 above contains three groups of strip convolution optimized convolutional modulation modules. The specific details are as follows:

[0078]

[0079] Where X and Y represent the input and output of the MCM module, respectively. Conv and DConv represent normal convolution and depthwise convolution, respectively. The subscripts represent the convolution kernel size, k = 11-2×(i-1), q = 1+2×(i-1), and i = 1, 2, and 3. The PGM module adopts the idea of deep supervision. First, based on the output feature Y of the MCM, a prediction head PreH is used to generate a change probability prediction map P. Then, P is used to adaptively adjust and enhance the feature Y, and feature fusion is performed through residual connections. The specific process is as follows:

[0080]

[0081] Where Y and Z represent the input and output of the PGM module respectively, and PreH represents the prediction head, which consists of multiple convolutional layers.

[0082] The multi-scale attention pyramid pooling module for the first-level features in Step 2 above is MAPPM-1. MAPPM-1 consists of two stages: multi-scale feature extraction and feature enhancement. In the first stage, six scale features are generated through six groups of pooling operations: a spatial dimension global average pooling GAPs extracts global image features, and five local average poolings of different sizes and strides extract local features of different scales of the image. Then, channel interaction and spatial dimension restoration are performed on each scale feature through convolution and upsampling operations. Finally, the six scale features are fused through cascading. The details are as follows:

[0083]

[0084] Where F represents the input of MAPPM-1, f i represents the i-th level feature of the pyramid, f c Represents cascade features, C represents cascade, Conv 1×1 represents 1×1 convolution, Up represents bilinear interpolation upsampling, GAPs represents global average pooling in spatial dimension, Avg (k,q) represents average pooling with size k×k and step size q;

[0085] In the feature enhancement stage, a multi-scale channel attention module is proposed to adaptively enhance features of different scales. A multi-scale channel attention module (MSCAM) is proposed. MSCAM consists of three branches: the first branch considers global context through global average pooling, the second branch considers local context through 3×3 pooling, and the third branch considers pixel-level context through 1×1 convolution. The specific details of MSCAM are as follows:

[0086]

[0087] Among them, GAPs represents the global average pooling of spatial dimensions, Avg (3,1)Indicates average pooling with a size of 3×3 and a step size of 1, Conv 1×1 represents 1×1 convolution, σ represents Sigmoid function; assuming that the output of MAPPM-1 is F ms , then F ms Finally, it is calculated by the following formula:

[0088] F ms =Conv 1×1 (C(f att ,Conv 1×1 (F)))+F;

[0089] Given that the receptive field of features at each level increases with the increase of the level, the proposed LDPPS uses MAPPM with decreasing pyramid levels: for the multi-scale pyramid pooling module MAPPM-2 of the second-level features, the maximum-scale local feature branch f5 in MAPPM-1 is removed; MAPPM-3, MAPPM-4 and MAPPM-5 of the 3rd to 5th level features can be obtained sequentially.

[0090] The global-local relationship perception feature fusion module GLRAF in Step 3 above consists of two stages: 1) modeling global relationships and 2) modeling local relationships. The specific processes of the two stages are as follows:

[0091] Step 3.1 Modeling global relationships; Calculate the global relationship matrix based on the channel attention of adjacent level features, using f s ∈r C×H×W and Represents shallow and deep features of adjacent levels;

[0092] Step 3.1.1. Perform channel interaction through 1×1 convolution and adjust f d Channel makes it have the same s The same number of channels, followed by spatial global average pooling GAPs and 1×1 convolution to generate F s and F d The channel attention vector att s and att d ;Step3.1.2, vector att s and att d Reorganize into r×k two-dimensional matrices, denoted as G s and G d , and calculate the global relationship GR matrix R by matrix multiplication g ;

[0093] Step 3.1.3, generate feature F s and F d The global enhancement weight M i(i=s or d), and for F s and F d Perform adaptive enhancement;

[0094] Step 3.2 Modeling local relationships; modeling enhanced features F by calculating the correlation between each pixel es and F ed Local relation LR;

[0095] Step 3.2.1, use bilinear interpolation to calculate F ed Upsample to make it consistent with F es With the same spatial resolution, F is generated by global maximum pooling GMPc and global average pooling GAPc in the channel dimension es and F ed The channel dimension is 2-dimensional matrix L s and L d ;

[0096] Step 3.2.2, calculate the local relationship matrix R l ;

[0097] Step 3.2.3, for feature F es and F ed Perform adaptive enhanced fusion.

[0098] The specific calculation process of the above Step 3.1 modeling global relationship is as follows:

[0099] Step 3.1.1. Perform channel interaction through 1×1 convolution and adjust F d Channel makes it have the same s The same number of channels, followed by spatial global average pooling GAPs and 1×1 convolution to generate F s and F d The channel attention vector att s and att d :

[0100] att i =Conv 1×1 (GAPs(Conv 1×1 (F i )))i=sord;

[0101] Step 3.1.2, vector att s and att d Reorganize into r×k two-dimensional matrices, denoted as G s and G d , and calculate the global relationship GR matrix R by matrix multiplication g :

[0102]

[0103] Step 3.1.3, generate feature F s and F d The global enhancement weight M i (i=s or d), and for F s and F d Perform adaptive enhancement:

[0104]

[0105] Among them F ei (i=s or d) represents the enhanced features, Reshape represents the transformation matrix dimension, and σ represents the sigmoid function.

[0106] The specific calculation process of the above Step 3.2 modeling local relationship is as follows:

[0107] Step 3.2.1, use bilinear interpolation to calculate F ed Upsample to make it consistent with F es With the same spatial resolution, F is generated by global maximum pooling GMPc and global average pooling GAPc in the channel dimension es and F ed The channel dimension is 2-dimensional matrix L s and L d :

[0108] L i =C(GMPc(F ei ),GAPc(F ei ))i=sord;

[0109] Step 3.2.2, calculate the local relationship matrix R l :

[0110] R l =σ(L s1 iL d1 +L s2 iL d2 );

[0111] Where L sk and L dk Represents the matrix L s and L d The kth (k=1 or 2) channel dimension of , σ represents the sigmoid function;

[0112] Step 3.2.3, for feature F es and F ed Perform adaptive enhancement fusion:

[0113] F fusion =αiR l iF es +βiR l iF ed ;

[0114] Where α and β represent two learnable parameters.

[0115] The prediction head of the fuzzy weighted enhancement in Step 3 above regards the classification probability of the pixel as a fuzzy set, estimates the classification reliability of each pixel by calculating the fuzziness of the fuzzy set, and proposes a fuzzy weighted cross entropy loss function:

[0116] Assume p ic and p iu denotes the predicted probability of pixel i belonging to the changed class and the unchanged class (p ic +p iu =1), then the set {p ic ,p iu} is regarded as a fuzzy set, so its fuzziness can be calculated by the following formula to estimate the classification uncertainty Fu of pixel i i :

[0117]

[0118] Fu i The larger the value of is, the lower the classification reliability of pixel i is, and the more attention should be paid to pixel i; therefore, based on Fu i The loss weight w of pixel i is calculated by the following formula i :

[0119] w i =2*σ(Fu i -Fu);

[0120] Where Fu represents the average classification uncertainty of the pixel, σ represents the sigmoid function; the fuzzy weighted cross entropy loss function CE fuzz Defined by the following formula:

[0121]

[0122] Where log represents the natural logarithm operator, p ic represents the predicted probability that pixel i belongs to the change class, p iu represents the predicted probability that pixel i belongs to the unchanged class, y ic Is a Boolean variable. If pixel i belongs to the change class, then y ic =1, otherwise y ic =0;

[0123] To increase the diversity of the loss, the proposed ambiguity-weighted cross entropy loss function CE is used in the final decoding stage. fuzz , in the deep supervision of the encoding stage, the traditional cross entropy loss CE is adopted; the total loss is as follows:

[0124]

[0125] Among them CE l represents the traditional cross entropy loss of layer l in deep supervision.

[0126] Example 1:

[0127] A. Overall structure of TSMSNet

[0128] like Figure 1 As shown in the figure, the proposed two-stage multi-scale change detection network TSMSNet adopts a U-shaped network architecture, which mainly consists of three parts: an encoder based on PGMCM, a skip connection structure based on LDPPS, and a decoder based on GL RAF.

[0129] In the encoding stage, the two phases of remote sensing images are first cascaded as network input; then they pass through 5 encoding levels in sequence to fully extract the features of the two phases of images. For 1-3 encoding levels, each level contains a traditional convolution block and a probabilistic gated multi-scale convolutional modulation module PGMCM. The traditional convolution block (denoted as CBR) contains convolution Conv, batch normalization BN and Relu activation operations in sequence. PGMCM includes two sub-modules: multi-scale convolutional modulation module MCM and probabilistic gated module PGM. For 4-5 encoding levels, each level contains a CBR module and an MCM module. For each encoding level, CBR is first used to perform preliminary extraction of image features, and then PGMCM (or MCM) is used to fully extract the change features of different scales.

[0130] In skip connections, by considering the receptive field of each level's features, a hierarchical decreasing pyramid pooling structure (LDPPS) is proposed to further extract features of varying scales. Unlike existing U-shaped networks, which typically perform multi-scale pooling only on the highest-level features, LDPPS implements a multi-scale attention pyramid pooling module (MAPPM) for each level of features. The number of MAPPM layers decreases as the receptive field of the hierarchical features increases. Furthermore, MAPPM employs a multi-scale attention scheme to adaptively enhance features of varying scales at each level.

[0131] The decoding phase primarily consists of four GLRAF modules and a prediction head with ambiguity weighting. The GLRAF module effectively fuses features from different levels by comprehensively considering the global and local relationships between features at adjacent levels, enabling more efficient feature decoding and resolution reconstruction. Ambiguity weighting allows the model to better focus on classifying difficult samples, improving change detection accuracy.

[0132] B. Probabilistic Gated Multi-Scale Convolutional Modulation Module PGMCM

[0133] In recent years, a new feature learning paradigm, convolutional modulation, has received increasing attention. Convolutional modulation fuses features from different subspaces through pixel-by-pixel multiplication, implicitly mapping low-dimensional spatial features to high-dimensional space, significantly improving the network's feature representation capabilities. Figure 2 (a) A typical convolution modulation module is given, which includes two convolution branches, each mapping the image features to a different feature space, and then fusing the two features by pixel-by-pixel multiplication. The specific calculation formula is as follows:

[0134] Z=DConv k×k (Conv 1×1 (X))iConv 1×1 (X); (1)

[0135] Where X and Z represent the input and output of convolution modulation respectively, Conv 1×1 Represents 1×1 convolution, DConv k×k represents a k×k depthwise convolution, and represents pixel-by-pixel multiplication.

[0136] Convolutional modulation is similar to a kernel function to a certain extent. It can significantly increase the dimension of the feature space and enhance the network's ability to represent features without explicitly increasing the network width. However, the convolutional modulation module given by Equation (1) uses a set of convolution kernels of specific sizes, which makes its receptive field relatively simple and cannot fully extract features of multiple scales. At the same time, its use of a k×k square convolution kernel, although very beneficial for extracting low-frequency information in the image, is not conducive to extracting high-frequency information such as boundary contours.

[0137] To solve the above problems and highlight the change characteristics, this paper proposes a probabilistic gated multi-scale convolution modulation module PGMCM based on convolution modulation, strip convolution and gating mechanism. Figure 2As shown in (b): First, PGMCM uses three groups of convolutional modulation modules in parallel, and each group uses convolution kernels of complementary sizes (for example, the first group uses 1×1 and 11×11, and the second group uses 7×7 and 5×5 convolution kernels) to fully extract change features of different scales. Secondly, to effectively enhance detailed features such as change boundaries, PGMCM introduces strip convolutions of different sizes and directions (such as 1×3 and 5×1 convolutions) into the convolutional modulation module. Strip convolution uses strip convolution kernels, which can more effectively extract boundary information. Finally, to suppress background information and highlight change information, PGMCM uses deep supervision and gating mechanisms to dynamically control the information flow of features using the predicted change probability map generated based on the current features.

[0138] like Figure 2 As shown in (b), PGMCM consists of two submodules: the multi-scale convolutional modulation module MCM and the probabilistic gating module PGM. MCM contains three groups of convolutional modulation modules optimized by strip convolution. The details are as follows:

[0139]

[0140] Where X and Y represent the input and output of the MCM module, respectively. Conv and DConv represent normal convolution and depthwise convolution, respectively. The subscripts represent the convolution kernel size, k = 11-2×(i-1), q = 1+2×(i-1), and i = 1, 2, and 3. The PGM module uses deep supervision. First, based on the output feature Y of the MCM, a prediction head PreH is used to generate a change probability prediction map P. Then, P adaptively adjusts and enhances the feature Y, and performs feature fusion through residual connections. The specific process is as follows:

[0141]

[0142] Where Y and Z represent the input and output of the PGM module respectively, and PreH represents the prediction head, which consists of multiple convolutional layers.

[0143] The Probabilistic Gated Multi-Scale Convolutional Modulation Module (PGMCM) offers the following advantages: 1) Based on convolutional modulation, a new paradigm for feature learning with strong feature extraction capabilities, it implicitly maps features from low-dimensional space to high-dimensional space. 2) Utilizing multiple groups of convolutional modulation modules, it effectively extracts features of varying scales; introducing stripe convolution into the convolutional modulation module effectively enhances boundary features. 3) Using a probabilistic gating mechanism, it regulates information flow, enhancing varying information and suppressing background information.

[0144] C. Hierarchical Descending Pyramid Pooling Structure LDPPS

[0145] Spatial pyramid pooling is a classic method for acquiring multi-scale features and is widely used in the field of change detection. Although the pyramid pooling structure can effectively extract features at multiple scales and help detect changing objects at different scales, existing change detection networks often perform pyramid pooling based on the last level of features in the encoding phase. This has the following shortcomings: 1) The last level of features has a smaller resolution (1 / 32 or 1 / 16), corresponding to a larger receptive field, making pyramid pooling based on the last level of features unfavorable for extracting small-scale changing objects; 2) Considering only the last level of features fails to fully extract information about changes at multiple scales.

[0146] To solve the above problems, the present invention proposes a hierarchical decreasing pyramid pooling structure LDPPS by considering the receptive field of hierarchical features. Figure 1 LDPPS has the following advantages: 1) A multi-scale attention pyramid pooling module (MAPPM) is designed for each level of features to fully extract the multi-scale features of remote sensing images; 2) The number of pyramid levels of MAPPM decreases as the feature receptive field increases, which can better adapt to features at different levels. Figure 3 A multi-scale attention pyramid pooling module (MAPPM-1) for first-level features is presented. This paper takes MAPPM-1 as an example to introduce the specific details of MAPPM.

[0147] like Figure 3 As shown in the figure, MAPPM-1 mainly consists of two stages: multi-scale feature extraction and feature enhancement. In the first stage, six scale features are generated through six groups of pooling operations: a spatial dimension global average pooling GA Ps extracts global image features, and five local average poolings of different sizes and steps extract local features of different scales of the image; then, channel interaction and spatial dimension restoration are performed on each scale feature through convolution and upsampling operations; finally, the six scale features are fused through cascading. The details are as follows:

[0148]

[0149] Where F represents the input of MAPPM-1, f i represents the i-th level feature of the pyramid, f c Represents cascade features, C represents cascade, Conv 1×1 represents 1×1 convolution, Up represents bilinear interpolation upsampling, GAPs represents global average pooling in spatial dimension, Avg (k,q) represents average pooling with size k×k and stride q.

[0150] In the feature enhancement stage, a multi-scale channel attention module is given to adaptively enhance features of different scales. In order to model the correlation between channels, the prior art "Squeeze-and-Excitation Networks" proposes a squeeze-excitation attention module SE. SE generates channel attention with the help of global average pooling, which can significantly improve the network's ability to represent features with only a small increase in computational cost. However, SE squeezes each channel feature map into a scalar value, making it more conducive to highlighting large-scale targets in the image, but easily ignoring small-scale targets. To address this problem, the present invention provides a multi-scale channel attention module MSCAM. MSCAM consists of three branches: the first branch is similar to SE, considering the global context through global average pooling, the second branch considers the local context through 3×3 pooling, and the third branch considers the pixel-level context through 1×1 convolution. MSCAM can effectively enhance its perception of small-size targets by adding two scales of local context. The specific details of MSCAM are as follows:

[0151]

[0152] Among them, GAPs represents the global average pooling of spatial dimensions, Avg (3,1) Indicates average pooling with a size of 3×3 and a step size of 1, Conv 1×1 represents 1×1 convolution, σ represents Sigmoid function. Assume that the output of MAPPM-1 is F ms , then F ms Finally, it is calculated by the following formula:

[0153] F ms =Conv 1×1 (C(f att ,Conv 1×1 (F)))+F ; (6)

[0154] Given that the receptive field of features at each level increases with increasing levels, the proposed LDPPS uses MAPPM with decreasing pyramid levels: for the multi-scale pyramid pooling module MAPPM-2 for the second-level features, simply remove the maximum-scale local feature branch f5 from MAPPM-1. MAPPM-3, MAPPM-4, and MAPPM-5 for the third to fifth levels can be obtained similarly.

[0155] D. Global-local relationship perception feature fusion module

[0156] Existing change detection networks often fuse features from different layers during the decoding phase through concatenation, addition, or attention mechanisms. However, these methods assume that features from different layers have equal contributions, which is unreasonable: shallow features contain more spatial details, while deep features contain more global and semantic information, and features from different layers have different contributions.

[0157] The present invention proposes a global-local relationship perception feature fusion module GLRAF for the change detection task. GLRAF considers the global and local relationships of features at adjacent levels at the same time, and can better fuse features at different levels. There are the following differences between GLRAF and RAF: 1) The former is aimed at the hierarchical feature fusion of change detection tasks, while the latter is aimed at the fusion of two encoding branch features of semantic segmentation tasks; 2) The former considers the global and local relationships of features at the same time, while the latter only considers the global relationship of features. Figure 5 As shown in Figure 3, the proposed GLRAF consists of two stages: 1) modeling global relations; 2) modeling local relations.

[0158] The first stage is similar to RAF, which calculates the global relationship matrix based on the channel attention of adjacent level features. s ∈R C×H×W and Represents the shallow features and deep features of adjacent layers. First, channel interaction is performed through 1×1 convolution and F is adjusted d Channel makes it have the same s The same number of channels, followed by spatial global average pooling GAPs and 1×1 convolution to generate F s and F d The channel attention vector att s and att d :

[0159] att i =Conv 1×1 (GAPs(Conv 1×1 (F i )))i=s ord; (7)

[0160] Then the vector att s and att d Reorganize into r×k two-dimensional matrices, denoted as G s and G d , and calculate the global relationship GR matrix R by matrix multiplication g :

[0161]

[0162] Where T represents the transposition operation. Finally, the feature F is generated by formula (9) sand F d The global enhancement weight M i (i=sor d), and for F s and F d Perform adaptive enhancement:

[0163]

[0164] Among them F ei (i=s or d) represents the enhanced features, Reshape represents the transformation matrix dimension, and σ represents the sigmoid function.

[0165] The second stage is to calculate the correlation between each pixel and model the enhanced feature F es and F ed The local relation LR of F is firstly obtained by bilinear interpolation. ed Upsample to make it consistent with F es With the same spatial resolution, F is generated by global maximum pooling GMPc and global average pooling GAPc in the channel dimension es and F ed The channel dimension is 2-dimensional matrix L s and L d :

[0166] L i =C(GMPc(F ei ),GAPc(F ei ))i=s or d; (10)

[0167] Then the local relationship matrix R is calculated by the following formula l :

[0168] R l =σ(L s1 iL d1 +L s2 iL d2 ); (11)

[0169] Where L sk and L dk Represents the matrix L s and L d The kth (k=1or 2) channel dimension of , σ represents the sigmoid function. Finally, the feature F is calculated by formula (12) es and F ed Perform adaptive enhanced fusion.

[0170] F fusion =αiR l iF es +βiR l iF ed; (12)

[0171] Where α and β represent two learnable parameters.

[0172] E. Ambiguity Weighted Cross Entropy Loss Function

[0173] In the change detection map, each pixel has a different classification reliability. For example, the transition from the changed class to the unchanged class is a gradual process. The boundary between the changed class and the unchanged class is usually not an absolute line, but a fuzzy area with a certain width; the classification reliability of pixels in the category boundary area is generally lower than the classification reliability of pixels within the category. However, when calculating the loss, the existing change detection network usually treats all pixels equally and does not fully consider the classification reliability of the pixels. In response to the above problems, the present invention regards the classification probability of the pixel as a fuzzy set, estimates the classification reliability of each pixel by calculating the fuzziness of the fuzzy set, and proposes a fuzziness-weighted cross entropy loss function. By using the proposed loss function, the network can pay more attention to samples that are difficult to classify, which is expected to improve the performance of significant change detection, especially for the boundary area of the changed target.

[0174] Assume p ic and p iu denotes the predicted probability of pixel i belonging to the changed class and the unchanged class (p ic +p iu =1), then the set {p ic ,p iu} is regarded as a fuzzy set, so its fuzziness can be calculated by the following formula to estimate the classification uncertainty Fu of pixel i i :

[0175]

[0176] Generally speaking, Fu i The larger the value of , the lower the classification reliability of pixel i, and the more attention should be paid to pixel i. i The loss weight w of pixel i is calculated by the following formula i :

[0177] w i =2*σ(Fu i -Fu); (14)

[0178] Where Fu represents the average classification uncertainty of the pixel, σ represents the sigmoid function. Fuzzy weighted cross entropy loss function CE fuzz Defined by the following formula:

[0179]

[0180] Where log represents the natural logarithm operator, p icrepresents the predicted probability that pixel i belongs to the change class, p iu represents the predicted probability that pixel i belongs to the unchanged class, y ic Is a Boolean variable. If pixel i belongs to the change class, then y ic =1, otherwise y ic =0.

[0181] To increase the diversity of loss, this paper only uses the proposed ambiguity-weighted cross entropy loss function CE in the final decoding stage. fuzz , in the deep supervision of the encoding stage, the traditional cross entropy loss CE is used. The total loss is as follows:

[0182]

[0183] Among them CE l represents the traditional cross entropy loss of layer l in deep supervision.

[0184] Experimental results and analysis

[0185] A. Dataset

[0186] To validate the effectiveness of the proposed change detection network, TSMSNet, experiments were conducted using three commonly used public change detection datasets: WHU (Fully Convolutional Networks for Multisource Building Extraction From an Open Aerial and Satellite Imagery Data Set), GZ (ASemisupervised Convolutional Neural Network for Change Detection in High-Resolution Remote-Sensing Images), and GVLM (Cross-domain landslide mapping from large-scale remote sensing images using prototype-guided domain-aware progressive representation learning). WHU and GZ focus on building changes, while GVLM focuses on landslide changes.

[0187] The WHU dataset consists of a pair of remote sensing images with a size of 32,507 × 15,354 pixels and a spatial resolution of 0.2 meters. These images were acquired in 2012 and 2016, respectively, covering the city of Christchurch, New Zealand. The GZ dataset contains 19 pairs of seasonally varying remote sensing images with sizes ranging from 1,006 × 1,168 pixels to 4,936 × 5,224 pixels and a spatial resolution of 0.55 meters. These images were acquired between 2006 and 2019 and cover a suburban area in China. The GVLM dataset consists of 17 pairs of remote sensing images with sizes ranging from 1,006 × 1,168 pixels to 4,936 × 5,224 pixels and a spatial resolution of 0.59 meters. These images were acquired between 2010 and 2021 and cover landslide sites in different countries and regions across six continents: Asia, Africa, North America, South America, Europe, and Oceania.

[0188] Considering computational resource constraints, the images were divided into 256×256 pixel patches using non-overlapping cropping. The patches in each dataset were randomly split into training, validation, and test sets using a 7:1:2 ratio. The datasets were augmented using flipping, scaling, cropping, and Gaussian blurring.

[0189] B. Experimental Environment Setup

[0190] In order to test the performance of the present invention and combine its own characteristics, the following comparative experiments were organized: 1) comparison with three groups of attention-improved CNN change detection networks, namely IFN, SNUNet and DMINet; 2) comparison with three groups of Transformer-integrated CNN change detection networks, namely BIT, MSCANet and STADE-CDNet.

[0191] Four sets of accuracy indicators commonly used in change detection literature are used for quantitative evaluation, namely: Precision, Recall, Intersection over Union (IoU), and F1 value.

[0192] All experiments were implemented based on the PyTorch framework, using an NVIDIA RTX 3060 graphics card with 12GB of video memory. The model parameters were trained using the SGD optimizer, with weight decay set to 0.0005, and the learning rate was updated using LinearLR, with an initial learning rate of 0.01 and a final learning rate of 0.0000995. The initial learning rate was decayed by 0.01% each round. The batch size is 8 and the maximum number of training epochs is 200.

[0193] C. Analysis of experimental results

[0194] Figures 5 to 7The change detection results for five typical regions on the WHU, GZ, and GVLM datasets are shown: (a) to (c) show the images at time T1, the images at time T2, and the ground truth labels, respectively; (d) to (j) show the change detection images generated by the networks IFN, SNUNet, DMINet, BIT, MSCANet, STADE-CDNet, and our proposed method TSMSNet, respectively. Red areas indicate false detections, green areas indicate missed detections, white areas indicate changed regions, and black areas indicate unchanged regions.

[0195] from Figure 5 As can be seen, for the WHU data targeting building changes, the proposed change detection network, TSMSNet, produces the change detection map that is closest to the ground-truth label map, containing the smallest colored areas. TSMSNet detects building changes of varying sizes and types relatively completely, with more accurate boundaries. The six compared methods perform less well. For example, for the larger building changes in the first and second rows, the change detection maps of the six compared methods all exhibit significant green missed detection errors, while the proposed TSMSNet accurately detects the changed areas. For the smaller building changes in the third row, the change detection maps of IFN, SNUNet, and DMINet exhibit significant false detection errors, while BIT, MSCANet, and STADE-CDNet exhibit significant missed detection errors. The change detection map of our proposed method, TSMSNet, only exhibits minor missed detection errors. For the fifth row, the change detection maps of IFN, SNUNet, DMINet, BIT, and MSCANet exhibit significant false detections, while STADES-CDNet significantly improves on this issue. However, STADES-CDNet's detection results are still less than ideal compared to our proposed method.

[0196] from Figure 6 As can be seen, the proposed TSMSNet achieves optimal change detection results on the GZ building change data. For example, for the smaller building changes within the rectangular box in the first row, all six comparison methods miss significant building changes, while our method, TSMSNet, significantly reduces these errors. For the building boundaries within the rectangular box in the second row, our method, TSMSNet, significantly outperforms the six baseline methods. For the third row, while all seven methods experience missed detections, our method achieves significantly smaller missed areas compared to the six comparison methods.

[0197] from Figure 7As can be seen, the proposed change detection network, TSMSNet, also achieves optimal change detection results for irregularly shaped GVLM landslide change data. For example, in row 1, the six comparison methods all missed the vast majority of landslide changes, while TSMSNet significantly reduced missed detection errors. For rows 2-5, TSMSNet detects more complete landslide changes and more accurate boundaries than the six comparison methods.

[0198] To more objectively evaluate the performance of different change detection methods, Table 1 presents quantitative evaluation metrics for the change detection results of different methods on three data sets. As shown in Table 1, our proposed method, TSMSNet, achieves the best accuracy on all three datasets. For both the WHU and GZ data, TSMSNet achieves the best values for Pre, Rec, F1, and Intersection over Union (IoU). For example, for the WHU data, TSMSNet achieves IoU and F1 values of 83.07% and 90.75%, respectively, which are at least 5.57% and 3.43% higher than those of other methods. For the GZ data, TSMSNet achieves IoU and F1 values of 73.39% and 84.65%, respectively, which are at least 2.57% and 1.7% higher than those of other methods. For the GVLM data, the proposed change detection network, TSMSNet, achieves the best results for Rec, F1, and IoU, but achieves the second-best result for Pre. For example, TSMSNet achieves an IoU value of 79.65%, which is at least 1.05% higher than that of other methods.

[0199] Table 1 Comparison results on three datasets

[0200]

[0201] D. Ablation Experiment

[0202] The proposed change detection network contains four innovations: probabilistic gated multi-scale convolutional modulation module PGMCM, hierarchical decreasing pyramid pooling structure LDPPS, feature fusion module GLRAF based on global-local correlation and ambiguity weighted cross entropy loss function CE. fuzz This section uses WHU and GVLM data as an example to analyze PGMCM, LDPPS, GLRAF and CE fuzz The first line uses the ordinary convolution module to replace the PGMCM change detection result; the second line removes the LDPPS change detection result; the third line uses the cascade fusion to replace the GLRAF change detection result; the fourth line uses the traditional cross entropy loss function to replace the CE fuzz The change detection results obtained are shown in Table 2. The fifth row shows the change detection results of TSMSNet.

[0203] Table 2 Network structure ablation experiment results

[0204]

[0205] By comparing the first and fifth rows, we can see that the proposed PGMCM module can effectively improve the change detection accuracy of TSMSNet. For example, for WHU data, PGMCM increases the IoU and F1 values of TSMSNet by 1.41% and 0.85% respectively. By comparing the second and fifth rows, we can see that the proposed LDPPS structure can significantly enhance the change detection performance of TSMSNet. For example, for GVLM data, LDPPS increases the IoU and F1 values of TSMSNet by 2.82% and 1.78% respectively. By comparing the third and fifth rows, we can see that the proposed GLRAF module can effectively improve the change detection effect of TSMSNet. For example, for WHU data, GLRAF increases the IoU and F1 values of TSMSNet by 1.18% and 0.71% respectively. By comparing the fourth and fifth rows, we can see that the proposed loss function CE fuzz It can also enhance the change detection performance of TSMSNet. For example, for WHU data, CE fuzz The IoU and F1 values of TSMSNet are increased by 0.82% and 0.49% respectively. From the above analysis, we can see that PGMCM, LDPPS, GLRAF and CE fuzz Both can effectively improve the accuracy of change detection.

Claims

1. A two-stage multi-scale change detection method for high-resolution remote sensing images, characterized by: The steps of change detection are: Step 1: The encoder based on PGMCM concatenates two remote sensing images as network input. It then passes through n encoding levels in sequence to extract the features of the two images. The encoding levels are downsampled to the next level, and the extraction results of each level are output to the jump connection structure based on LDPPS. Step 2: The LDPPS-based skip connection structure is sequentially connected to multiple encoding levels of the PGMCM-based encoder. By considering the receptive field of each level feature, a hierarchical decreasing pyramid pooling structure LDPPS is formed to further extract the changing features of different scales. The hierarchical decreasing pyramid pooling structure LDPPS provides a multi-scale attention pyramid pooling module MAPPM for each level feature. The number of MAPPM layers decreases step by step as the receptive field of the level feature increases. MAPPM adaptively adjusts and enhances the features of different scales at each level through multi-scale attention and outputs the results to the GLRAF-based decoder. Step 3. The GLRAF-based decoder contains n-1 global-local relationship perception feature fusion modules GLRAF and a fuzzy weighted enhanced prediction head. The n-1 GLRAF modules are directly connected to the n-1 level outputs of LDPPS from top to bottom. The output of the bottom layer of LDPPS, i.e. the nth layer, is connected to the GLRAF module of the adjacent upper layer through upsampling. The GLRAF modules are connected upward through upsampling. The top GLRAF module outputs the detection result through the fuzzy weighted enhanced prediction head.

2. The dual-stage multi-scale change detection method for high-resolution remote sensing images according to claim 1, characterized in that: Said n=5.

3. The dual-stage multi-scale change detection method for high-resolution remote sensing images according to claim 2, characterized in that: The PGMCM-based encoder has five coding levels. For the first 1-3 coding levels, each level includes a traditional convolution block CBR and a probability gated multi-scale convolution modulation module PGMCM; the traditional convolution block CBR includes convolution Conv, batch normalization BN and Relu activation function in sequence; the probability gated multi-scale convolution modulation module PGMCM includes two sub-modules: a multi-scale convolution modulation module MCM and a probability gated module PGM; for the 4th-5th coding levels, each level includes a traditional convolution block CBR and a multi-scale convolution modulation module MCM.

4. The dual-stage multi-scale change detection method for high-resolution remote sensing images according to claim 3, characterized in that: The probability gated multi-scale convolutional modulation module PGMCM in Step 1 includes three groups of strip convolution optimized convolutional modulation modules. The specific details are as follows: Where X and Y represent the input and output of the MCM module, respectively. Conv and DConv represent ordinary convolution and depthwise convolution, respectively. The subscripts represent the convolution kernel size, k = 11-2×(i-1), q = 1+2×(i-1), and i = 1, 2, and 3. The PGM module adopts the idea of deep supervision. First, based on the output feature Y of the MCM, a prediction head PreH is used to generate a change probability prediction map P. Then, P is used to adaptively adjust and enhance the feature Y, and feature fusion is performed through residual connections. The specific process is as follows: Where Y and Z represent the input and output of the PGM module respectively, and PreH represents the prediction head, which consists of multiple convolutional layers.

5. The dual-stage multi-scale change detection method for high-resolution remote sensing images according to claim 4, characterized in that: The multi-scale attention pyramid pooling module for the first-level features in Step 2 is MAPPM-1. MAPPM-1 consists of two stages: multi-scale feature extraction and feature enhancement. In the first stage, six scale features are generated through six groups of pooling operations: a spatial dimension global average pooling GAPs extracts global image features, and five local average poolings of different sizes and steps extract local features of different scales of the image. Then, channel interaction and spatial dimension restoration are performed on each scale feature through convolution and upsampling operations. Finally, the six scale features are fused through cascading. The details are as follows: Where F represents the input of MAPPM-1, f i represents the i-th layer feature of the pyramid, f c Represents cascade features, C represents cascade, Conv 1×1 represents 1×1 convolution, Up represents bilinear interpolation upsampling, GAPs represents global average pooling in spatial dimension, Avg (k,q) represents average pooling with size k×k and step size q; In the feature enhancement stage, a multi-scale channel attention module is proposed to adaptively enhance features of different scales. A multi-scale channel attention module (MSCAM) is proposed. MSCAM consists of three branches: the first branch considers global context through global average pooling, the second branch considers local context through 3×3 pooling, and the third branch considers pixel-level context through 1×1 convolution. The specific details of MSCAM are as follows: Among them, GAPs represents the global average pooling of spatial dimensions, Avg (3,1) Indicates average pooling with a size of 3×3 and a step size of 1, Conv 1×1 represents 1×1 convolution, σ represents Sigmoid function; assuming that the output of MAPPM-1 is F ms , then F ms Finally, it is calculated by the following formula: F ms =Conv 1×1 (C(f att ,Conv 1×1 (F)))+F; Given that the receptive field of features at each level increases with the increase of the level, the proposed LDPPS uses MAPPM with decreasing pyramid levels: for the multi-scale pyramid pooling module MAPPM-2 of the second-level features, the maximum-scale local feature branch f5 in MAPPM-1 is removed; MAPPM-3, MAPPM-4 and MAPPM-5 of the 3rd to 5th level features can be obtained sequentially.

6. The dual-stage multi-scale change detection method for high-resolution remote sensing images according to claim 5, characterized in that: The global-local relationship perception feature fusion module GLRAF in Step 3 consists of two stages: The specific process of the two stages of 1) modeling global relationships and 2) modeling local relationships is as follows: Step 3.1 Modeling global relationships; Calculate the global relationship matrix based on the channel attention of adjacent level features, using F s ∈R C×H×W and Represents shallow and deep features of adjacent levels; Step 3.1.

1. Perform channel interaction through 1×1 convolution and adjust F d Channel makes it have the same s The same number of channels, followed by spatial global average pooling GAPs and 1×1 convolution to generate F s and F d The channel attention vector att s and att d ; Step 3.1.2, vector att s and att d Reorganize into r×k two-dimensional matrices, denoted as G s and G d , and calculate the global relationship GR matrix R by matrix multiplication g ; Step 3.1.3, generate feature F s and F d The global enhancement weight M i (i=s or d), and for F s and F d Perform adaptive enhancement; Step 3.2 Modeling local relationships; modeling enhanced features F by calculating the correlation between each pixel es and F ed Local relation LR; Step 3.2.1, use bilinear interpolation to calculate F ed Upsample to make it consistent with F es With the same spatial resolution, F is generated by global maximum pooling GMPc and global average pooling GAPc in the channel dimension es and F ed The channel dimension is 2-dimensional matrix L s and L d ; Step 3.2.2, calculate the local relationship matrix R l ; Step 3.2.3, for feature F es and F ed Perform adaptive enhanced fusion.

7. The dual-stage multi-scale change detection method for high-resolution remote sensing images according to claim 6, characterized in that: The specific calculation process of Step 3.1 modeling global relationships is as follows: Step 3.1.

1. Perform channel interaction through 1×1 convolution and adjust F d Channel makes it have the same s The same number of channels, followed by spatial global average pooling GAPs and 1×1 convolution to generate F s and F d The channel attention vector att s and att d : to i =Conv 1×1 (GAPs(Conv 1×1 (F i )))i=word; Step 3.1.2, vector att s and att d Reorganize into r×k two-dimensional matrices, denoted as G s and G d , and calculate the global relationship GR matrix R by matrix multiplication g : Step 3.1.3, generate feature F s and F d The global enhancement weight M i (i=s or d), and for F s and F d Perform adaptive enhancement: Among them F ei (i=s or d) represents the enhanced features, Reshape represents the transformation matrix dimension, and σ represents the sigmoid function.

8. The dual-stage multi-scale change detection method for high-resolution remote sensing images according to claim 7, characterized in that: The specific calculation process of the Step 3.2 modeling local relationship is as follows: Step 3.2.1, use bilinear interpolation to calculate F ed Upsample to make it consistent with F es With the same spatial resolution, F is generated by global maximum pooling GMPc and global average pooling GAPc in the channel dimension es and F ed The channel dimension is 2-dimensional matrix L s and L d : L i =C(GMPc(F ei ),GAPc(F ei ))i=s ord; Step 3.2.2, calculate the local relationship matrix R l : R l =σ(L s1 he d1 +L s2 he d2 ); Where L sk and L dk Represents the matrix L s and L d The kth (k=1 or 2) channel dimension of , σ represents the sigmoid function; Step 3.2.3, for feature F es and f ed Perform adaptive enhancement fusion: F fusion =αiR l iF es +βiR l iF ed ; Where α and β represent two learnable parameters.

9. The dual-stage multi-scale change detection method for high-resolution remote sensing images according to claim 8, characterized in that: The prediction head of the fuzzy weighted enhancement in Step 3 regards the classification probability of the pixel as a fuzzy set, estimates the classification reliability of each pixel by calculating the fuzziness of the fuzzy set, and proposes a fuzzy weighted cross entropy loss function: Assume p ic and p iu denotes the predicted probability of pixel i belonging to the changed class and the unchanged class (p ic +p iu =1), then the set {p ic ,p iu } is regarded as a fuzzy set, so its fuzziness can be calculated by the following formula to estimate the classification uncertainty Fu of pixel i i : Fu i The larger the value of is, the lower the classification reliability of pixel i is, and the more attention should be paid to pixel i; therefore, based on Fu i The loss weight w of pixel i is calculated by the following formula i : w i =2*σ(Fu i -Fu); Where Fu represents the average classification uncertainty of the pixel, σ represents the sigmoid function; the fuzzy weighted cross entropy loss function CE fuzz Defined by the following formula: Where log represents the natural logarithm operator, p ic represents the predicted probability that pixel i belongs to the change class, p iu represents the predicted probability that pixel i belongs to the unchanged class, y ic Is a Boolean variable. If pixel i belongs to the change class, then y ic =1, otherwise y ic =0; To increase the diversity of the loss, the proposed ambiguity-weighted cross entropy loss function CE is used in the final decoding stage. fuzz , in the deep supervision of the encoding stage, the traditional cross entropy loss CE is adopted; the total loss is as follows: Among them CE l represents the traditional cross entropy loss of layer l in deep supervision.

Images