Change detection method for heterogeneous remote sensing images based on topological structure coupling

Abstract

The present invention discloses a heterogeneous remote sensing image change detection method based on topological structure coupling. First, a heterogeneous remote sensing image change detection neural network framework is designed. The mutual conversion of heterogeneous image domains is realized through the network, so that the difference map is calculated in the same domain, and the difference map calculated in two different image domains is comprehensively used to generate the final change result map, which effectively improves the accuracy of change detection; secondly, the wavelet layer in the network can effectively capture the texture structure characteristics of the input image and highlight the structural information of the original image. Then, by introducing the channel attention mechanism and the spatial attention mechanism, more weights are allocated to the region of interest, and unnecessary features are suppressed. Through the organic combination of wavelet, channel attention mechanism and spatial attention mechanism, the network can focus on the texture structure information of interest and suppress the difference of images from different domains. Experimental results show that the present invention effectively improves the accuracy of change detection.

Description

Technical Field

[0001] The present invention relates to the field of remote sensing image processing, and in particular to a heterogeneous remote sensing image change detection method based on topological structure coupling that can effectively utilize multimodal data from different sensors. Background Art

[0002] The purpose of remote sensing change detection methods is to identify changes occurring on the earth by comparing two or more images acquired at different times. It has been widely used in many fields, such as land use and cover evaluation, urban growth monitoring, and natural disaster assessment.

[0003] Heterogeneous remote sensing images refer to remote sensing images obtained from different sensors. Two heterogeneous images have different domains, different statistical distributions, inconsistent features, and significantly different appearance characteristics. Using heterogeneous remote sensing images for change detection is of great significance for the timely assessment of emergency disasters. It can not only greatly reduce the response time of the image processing system required for disaster management, but also achieve data complementarity. With the increase of different types of satellite sensors and the rapid development of remote sensing technology, the use of heterogeneous remote sensing images for change detection has shown more important practical significance.

[0004] Jensen et al. proposed a post-classification comparison (PCC) method based on unsupervised clustering in 1987 to detect wetland changes in heterogeneous images. In PCC, the pixels of multi-temporal heterogeneous images are first divided into different categories, such as wetlands, forests, and rivers, to obtain corresponding classification maps. Then, the classification maps are compared to generate change regions. Mubea and Menz developed the PCC method in 2012, using support vector machines (SVM) instead of unsupervised clustering for classification. However, the performance of the PCC method is easily affected by the classification accuracy. The results of change detection depend on whether the classification is accurate. The accumulation of classification errors will lead to a decrease in change detection performance. Wan et al. proposed a PCC method based on multi-temporal segmentation and composite classification and a PCC method based on cooperative multi-temporal segmentation and hierarchical composite classification in 2019. The use of multi-temporal segmentation methods to generate homogeneous objects can reduce the inevitable salt and pepper effect in pixel-based methods and reduce the errors caused by region conversion in traditional object-based methods. Then, composite classification is performed at the object level. This process exploits temporal correlation and overcomes the error propagation of traditional post-classification comparison methods. However, image segmentation affects the accuracy of change detection, especially for SAR images, where image segmentation is particularly difficult, which in turn affects the accurate detection of changes.

[0005] In recent years, the continuous development of deep learning has brought new ideas to remote sensing image processing. People have applied it to change detection of heterogeneous remote sensing images to improve the detection accuracy to a certain extent. Zhang et al. proposed a method based on stacked denoising autoencoders (SDAEs) in 2016. This method uses the invariant feature pairs selected from rough differential images to adjust the network parameters. However, the rough differential images are obtained manually or through existing algorithms, which makes the selection of invariant feature pairs affected by the performance of the differential image acquisition algorithm; Liu et al. proposed a symmetric convolutional coupled network (SCCN) method based on heterogeneous optical and SAR images in 2016. The symmetric network is used to convert two heterogeneous images into feature space to make the feature representation more consistent, generate the final detection image in the feature space, and learn the network parameters by optimizing the coupling function. However, this method ignores the impact of regional changes and does not perform changes in certain locations. Distinction; Niu et al. proposed a method based on conditional generative adversarial network (cGAN) in 2018 to realize image conversion. This method converts the optical image into SAR feature space, and directly compares the converted image with the approximate SAR image to obtain the final change result map. However, in the process of conversion, some features will be lost, resulting in some changes being missed; Wu et al. proposed a classification adversarial network in 2020. This method learns the relationship between heterogeneous image data and corresponding labels through adversarial training. The well-trained generator can be used to process the original heterogeneous image to obtain the final change detection result. However, the iterative training between the generator and the discriminator must find a suitable balance point, and the training is prone to collapse.

[0006] In general, existing methods generally only focus on the overall transformation of heterogeneous images, but lack the mining and application of image texture structure information, which to a certain extent affects the improvement of detection accuracy; in addition, most existing methods assign balanced weights to different regions of the ground objects, while the particularity and importance of the characteristics of the change points as the regions of interest have not been fully valued, and there is a lack of adaptive allocation of more weights to the regions of interest or features, which also restricts the improvement of detection accuracy to a certain extent. Summary of the invention

[0007] In order to solve the above technical problems existing in the prior art, the present invention provides a heterogeneous remote sensing image change detection method based on topological structure coupling, which can effectively highlight the texture structure information between heterogeneous remote sensing images and reduce the interference caused by different image domains.

[0008] The technical solution of the present invention is: a heterogeneous remote sensing image change detection method based on topological structure coupling, which is carried out in the following steps:

[0009] Step 1. Establish and initialize the convolutional neural network N for heterogeneous remote sensing image change detection hic , the N hic Contains 2 encoder sub-networks N encx and N ency And 2 decoder sub-networks N decx and N decy ;

[0010] Step 2: Input the training set H of heterogeneous remote sensing images X and Y to the convolutional neural network N hic Perform unsupervised training;

[0011] Step 3. Input standardized heterogeneous remote sensing images and , and use the convolutional neural network N that has completed unsupervised training hic Complete change detection.

[0012] The step 1 is specifically as follows:

[0013] Step 1.1 Establish and initialize the encoder subnetwork N encx , the sub-encoder network N encx Contains 1 wavelet layer Wavelet1, 3 convolutional layers Conv1_0, Conv1_1 and Conv1_2, 1 channel_attention1 module and 1 spatial_attention1 module;

[0014] The wavelet layer Wavelet1 includes 1 wavelet operation, 1 activation operation and 1 Dropout operation, uses a two-dimensional Haar wavelet transform to decompose the image to form 1 low-frequency image and 3 high-frequency images, and selects a nonlinear activation function LeakyReLU with a parameter of 0.3 as the activation function and a Dropout operation with a parameter of 0.2 for operation;

[0015] Conv1_0 includes 1 convolution operation, 1 activation operation and 1 Dropout operation, wherein the convolution layer contains 50 convolution kernels of size 3×3, each convolution kernel performs convolution operation with a step size of 1 pixel, and uses the nonlinear activation function LeakyReLU with a parameter of 0.3 as the activation function and the Dropout operation with a parameter of 0.2 for operation;

[0016] The Conv1_1 includes 1 convolution operation, 1 activation operation and 1 Dropout operation, wherein the convolution layer contains 50 convolution kernels of size 3×3, each convolution kernel performs convolution operation with a step size of 1 pixel, and uses the nonlinear activation function LeakyReLU with a parameter of 0.3 as the activation function and the Dropout operation with a parameter of 0.2 for operation;

[0017] Conv1_2 includes 1 convolution operation and 1 activation operation, wherein the convolution layer contains 3 convolution kernels of size 3×3, each convolution kernel performs convolution operation with a step size of 1 pixel, and selects nonlinear activation function Tanh as activation function for operation;

[0018] The channel_attention1 module includes 1 group of GlobalAvgPool1 layers, 1 group of GlobalMaxPool1 layers, 2 groups of convolutional layers, namely Conv1_3 and Conv1_4, and uses the nonlinear activation function Sigmoid to calculate the summation result;

[0019] The GlobalAvgPool1 includes 1 layer of GlobalAveragePooling2D operation and 1 layer of Reshape operation;

[0020] The GlobalMaxPool1 includes 1 layer of GlobalMaxPooling2D operation and 1 layer of Reshape operation;

[0021] Conv1_3 includes 1 convolution operation and 1 activation operation, wherein the convolution layer contains 4 convolution kernels of size 1×1, each convolution kernel performs convolution operation with a step size of 1 pixel, and uses the nonlinear activation function Relu as the activation function for operation;

[0022] Conv1_4 includes one convolution operation, wherein the convolution layer contains 50 convolution kernels of size 1×1, and each convolution kernel performs convolution operation with a step size of 1 pixel;

[0023] The spatial_attention1 module includes 1 AvgPool1 layer, 1 MaxPool1 layer, a custom connection layer Concat1 and 1 convolution layer Conv1_5;

[0024] AvgPool1 contains one reduce_mean operation, which performs calculation along the third dimension;

[0025] The MaxPool1 contains one reduce_max operation, which performs calculation along the third dimension;

[0026] The custom connection layer Concat1 is used to connect two features;

[0027] Conv1_5 includes one convolution operation and one activation operation, wherein the convolution layer contains one convolution kernel of size 7×7, each convolution kernel performs convolution operation with a step size of 1 pixel, and uses nonlinear activation function Sigmoid as activation function for operation;

[0028] Step 1.2. Establish and initialize the encoding subnetwork N ency , the sub-network N ency Contains 1 wavelet layer Wavelet2, 3 convolutional layers Conv2_0, Conv2_1 and Conv2_2, 1 channel_attention2 module and 1 spatial_atention2 module;

[0029] The wavelet layer Wavelet2 includes 1 wavelet operation, 1 activation operation and 1 Dropout operation, uses a two-dimensional Haar wavelet transform to decompose the image to form 1 low-frequency image and 3 high-frequency images, and uses a nonlinear activation function LeakyReLU with a parameter of 0.3 as an activation function and a Dropout operation with a parameter of 0.2 for operation;

[0030] The Conv2_0 includes 1 convolution operation, 1 activation operation and 1 Dropout operation, wherein the convolution layer contains 50 convolution kernels of size 3×3, each convolution kernel performs convolution operation with a step size of 1 pixel, and uses the nonlinear activation function LeakyReLU with a parameter of 0.3 as the activation function and the Dropout operation with a parameter of 0.2 for operation;

[0031] The Conv2_1 includes 1 convolution operation, 1 activation operation and 1 Dropout operation, wherein the convolution layer contains 50 convolution kernels of size 3×3, each convolution kernel performs convolution operation with a step size of 1 pixel, and uses the nonlinear activation function LeakyReLU with a parameter of 0.3 as the activation function and the Dropout operation with a parameter of 0.2 for operation;

[0032] The Conv2_2 includes 1 convolution operation and 1 activation operation, wherein the convolution layer contains 3 convolution kernels of size 3×3, each convolution kernel performs convolution operation with a step size of 1 pixel, and selects the nonlinear activation function Tanh as the activation function for operation;

[0033] The channel_attention2 module includes 1 group of GlobalAvgPool2 layers, 1 group of GlobalMaxPool2 layers, 2 groups of convolutional layers, namely Conv2_3 and Conv2_4, and uses the nonlinear activation function Sigmoid to calculate the summation result;

[0034] The GlobalAvgPool2 includes 1 layer of GlobalAveragePooling2D operation and 1 layer of Reshape operation;

[0035] The GlobalMaxPool2 includes 1 layer of GlobalMaxPooling2D operation and 1 layer of Reshape operation;

[0036] Conv2_3 includes 1 convolution operation and 1 activation operation, wherein the convolution layer contains 4 convolution kernels of size 1×1, each convolution kernel performs convolution operation with a step size of 1 pixel, and uses the nonlinear activation function Relu as the activation function for operation;

[0037] Conv2_4 includes one convolution operation, wherein the convolution layer contains 50 convolution kernels of size 1×1, and each convolution kernel performs convolution operation with a step size of 1 pixel;

[0038] The spatial_attention2 module includes 1 AvgPool2 layer, 1 MaxPool2 layer, a custom connection layer Concat2 and 1 convolution layer Conv2_5;

[0039] The AvgPool2 contains 1 reduce_mean operation, which performs calculation along the third dimension;

[0040] The MaxPool2 contains 1 reduce_max operation, which performs calculation along the third dimension;

[0041] The custom connection layer Concat2 is used to connect two features;

[0042] Conv2_5 includes one convolution operation and one activation operation, wherein the convolution layer contains one convolution kernel of size 7×7, each convolution kernel performs convolution operation with a step size of 1 pixel, and uses nonlinear activation function Sigmoid as activation function for operation;

[0043] Step 1.3. Establish and initialize the decoding subnetwork N decx , the sub-network N decxContains 3 groups of convolutional layers, namely Conv3_0, Conv3_1 and Conv3_2, 1 group of channel_attention3 modules, 1 group of spatial_attention3 modules and 1 group of inverse wavelet layer Rewavelet1;

[0044] The Conv3_0 includes 1 convolution operation, 1 activation operation and 1 Dropout operation, wherein the convolution layer contains 50 convolution kernels of size 3×3, each convolution kernel performs convolution operation with a step size of 1 pixel, and uses the nonlinear activation function LeakyReLU with a parameter of 0.3 as the activation function and the Dropout operation with a parameter of 0.2 for operation;

[0045] The Conv3_1 includes 1 convolution operation, 1 activation operation and 1 Dropout operation, wherein the convolution layer contains 50 convolution kernels of size 3×3, each convolution kernel performs convolution operation with a step size of 1 pixel, and uses the nonlinear activation function LeakyReLU with a parameter of 0.3 as the activation function and the Dropout operation with a parameter of 0.2 for operation;

[0046] Conv3_2 contains 1 convolution operation, 1 activation operation and 1 Dropout operation, where the convolution layer contains C x ×4 convolution kernels of size 3×3, where C x is the number of channels of the input image, each convolution kernel performs convolution operation with a step size of 1 pixel, and uses the nonlinear activation function LeakyReLU with a parameter of 0.3 as the activation function and the Dropout operation with a parameter of 0.2 for operation;

[0047] The Rewavelet1 includes one layer of inverse wavelet operation and one layer of activation operation, wherein the inverse wavelet layer uses two-dimensional Haar wavelet inverse transform to reconstruct the image, and uses nonlinear activation function Tanh as activation function for operation;

[0048] The channel_attention3 module includes 1 group of GlobalAvgPool3 layers, 1 group of GlobalMaxPool3 layers, 2 groups of convolutional layers, namely Conv3_3 and Conv3_4, and uses the nonlinear activation function Sigmoid to calculate the summation result;

[0049] The GlobalAvgPool3 includes 1 layer of GlobalAveragePooling2D operation and 1 layer of Reshape operation;

[0050] The GlobalMaxPool3 includes 1 layer of GlobalMaxPooling2D operation and 1 layer of Reshape operation;

[0051] Conv3_3 includes 1 convolution operation and 1 activation operation, wherein the convolution layer contains 4 convolution kernels of size 1×1, each convolution kernel performs convolution operation with a step size of 1 pixel, and uses the nonlinear activation function Relu as the activation function for operation;

[0052] Conv3_4 includes one convolution operation, wherein the convolution layer contains 50 convolution kernels of size 1×1, and each convolution kernel performs convolution operation with a step size of 1 pixel;

[0053] The spatial_attention3 module includes 1 AvgPool3 layer, 1 MaxPool3 layer, a custom connection layer Concat3 and 1 convolution layer Conv3_5;

[0054] The AvgPool3 contains 1 reduce_mean operation, which performs calculation along the third dimension;

[0055] The MaxPool3 contains 1 reduce_max operation, which performs calculation along the third dimension;

[0056] The custom connection layer Concat3 is used to connect two features;

[0057] Conv3_5 includes one convolution operation and one activation operation, wherein the convolution layer contains one convolution kernel of size 7×7, each convolution kernel performs convolution operation with a step size of 1 pixel, and uses the nonlinear activation function Sigmoid as the activation function for operation;

[0058] Step 1.4. Establish and initialize the encoding subnetwork N decy , the sub-network N decy Contains 3 groups of convolutional layers, namely Conv4_0, Conv4_1 and Conv4_2, 1 group of channel_attention4 modules, 1 group of spatial_attention4 modules and 1 group of inverse wavelet layer Rewavelet2;

[0059] The Conv4_0 includes 1 convolution operation, 1 activation operation and 1 Dropout operation, wherein the convolution layer contains 50 convolution kernels of size 3×3, each convolution kernel performs convolution operation with a step size of 1 pixel, and uses the nonlinear activation function LeakyReLU with a parameter of 0.3 as the activation function and the Dropout operation with a parameter of 0.2 for operation;

[0060] The Conv4_1 includes 1 convolution operation, 1 activation operation and 1 Dropout operation, wherein the convolution layer contains 50 convolution kernels of size 3×3, each convolution kernel performs convolution operation with a step size of 1 pixel, and uses the nonlinear activation function LeakyReLU with a parameter of 0.3 as the activation function and the Dropout operation with a parameter of 0.2 for operation;

[0061] Conv4_2 includes 1 convolution operation, 1 activation operation and 1 Dropout operation, where the convolution layer contains C y ×4 convolution kernels of size 3×3, where C y is the number of channels of the input image, each convolution kernel performs convolution operation with a step size of 1 pixel, and uses the nonlinear activation function LeakyReLU with a parameter of 0.3 as the activation function and the Dropout operation with a parameter of 0.2 for operation;

[0062] The Rewavelet2 includes one layer of inverse wavelet operation and one layer of activation operation, wherein the inverse wavelet layer uses two-dimensional Haar wavelet inverse transform to reconstruct the image, and uses nonlinear activation function Tanh as activation function for operation;

[0063] The channel_attention4 module includes 1 group of GlobalAvgPool4 layers, 1 group of GlobalMaxPool4 layers, 2 groups of convolutional layers of Conv4_3 and Conv4_4, and uses the nonlinear activation function Sigmoid to calculate the summation result;

[0064] The GlobalAvgPool4 includes 1 layer of GlobalAveragePooling2D operation and 1 layer of Reshape operation;

[0065] The GlobalMaxPool4 includes 1 layer of GlobalMaxPooling2D operation and 1 layer of Reshape operation;

[0066] The Conv4_3 includes 1 convolution operation and 1 activation operation, wherein the convolution layer contains 4 convolution kernels of size 1×1, each convolution kernel performs convolution operation with a step size of 1 pixel, and uses the nonlinear activation function Relu as the activation function for operation;

[0067] Conv4_4 includes one convolution operation, wherein the convolution layer contains 50 convolution kernels of size 1×1, and each convolution kernel performs convolution operation with a step size of 1 pixel;

[0068] The spatial_attention4 module includes 1 AvgPool4 layer, 1 MaxPool4 layer, a custom connection layer Concat4 and 1 convolution layer Conv4_5;

[0069] The AvgPool4 contains 1 reduce_mean operation, which is calculated along the third dimension;

[0070] The MaxPool4 contains 1 reduce_max operation, which performs calculation along the third dimension;

[0071] The custom connection layer Concat4 is used to connect two features;

[0072] The Conv4_5 includes 1 convolution operation and 1 activation operation, wherein the convolution layer contains 1 convolution kernel of size 7×7, each convolution kernel performs convolution operation with a step size of 1 pixel, and uses the nonlinear activation function Sigmoid as the activation function for operation.

[0073] The step 2 is specifically as follows:

[0074] Step 2.1. Standardize the input remote sensing image X according to formula (1), (2), (3) to obtain Where X = {X i |i=1,...,C1},X i represents the i-th channel of X, represent The i-th channel of , C1 represents the number of channels of X;

[0075] A=mean(X i )+3×std(X i ) (1)

[0076]

[0077]

[0078] Among them, mean(·) represents the mean of the image, std(·) represents the standard deviation of the image, and max(·) represents the maximum value of the image;

[0079] Step 2.2. According to formula (4), (5), (6), the input remote sensing image Y is standardized to obtain Where Y = {Y j |j=1,...,C2},Y j represents the j-th channel of Y, represent The jth channel of , C2 represents the number of channels of Y;

[0080] B=mean(Y j )+3×std(Y j ) (4)

[0081]

[0082]

[0083] Among them, mean(·) represents the mean of the image, std(·) represents the standard deviation of the image, and max(·) represents the maximum value of the image;

[0084] Step 2.3. and Extract pixel point sets from and in express The a-th pixel point in express The ath pixel point in , m represents the total number of selected pixel points;

[0085] Step 2.4. Each pixel point is centered on Divide into a series of pixel blocks X of size 100×100 H1 ,by Each pixel point is centered on Divide into a series of pixel blocks Y of size 100×100 H1 ;

[0086] Step 2.5. X H1 and Y H1 Each pixel block in is randomly rotated z×90 degrees counterclockwise with its center point as the rotation center to obtain the pixel block set X H2 and Y H2 , where z represents a value randomly selected from the set {1, 213, 4};

[0087] Step 2.6. X H1 and Y H1 Each pixel block in is flipped upside down to obtain the pixel block set X H3 and Y H3 ;

[0088] Step 2.7. Order Will and As the training set of the change detection neural network, it is sent to the network for unsupervised training, where kind represents the pixel block in the training set, set the number of iterations iter←1, and execute steps 2.8 to 2.19;

[0089] Step 2.8 uses the encoding subnetwork N encx extract Features;

[0090] Step 2.8.1 Using the encoding subnetwork N encx The first layer of Wavelet1 Perform feature extraction to obtain texture information feature F Wavelet1 ;

[0091] Step 2.8.2 Use convolutional layers Conv1_0 and Conv1_1 to perform F Wavelet1 Perform feature extraction and obtain feature F Conv1_1 ;

[0092] Step 2.8.3 Use the channel_attention1 module to focus on the feature F Conv1_1 Performing a treatment, comprising the following steps (a1) to (c1);

[0093] (a1) Use the parallel GlobalAvgPool1 layer and GlobalMaxPool1 layer to perform feature Conv1_1 Perform pooling operations to obtain features F Globalavg1 and F Globalmax1 ;

[0094] (b1) Use convolutional layers Conv1_3 and Conv1_4 to sequentially process feature F Globalavg1 Perform feature extraction to obtain F Gavgout1 , using convolutional layers Conv1_3 and Conv1_4 to successively Globalmax1 Perform feature extraction to obtain F Gmaxout1 ;

[0095] (c1) for feature F Gavgout1 and F Gmaxout1 Perform a summation operation, and then use the nonlinear activation function Sigmoid to calculate the summation result to obtain the weight coefficient F c_a1 ;

[0096] Step 2.8.4: Set the weight coefficient F c_a1 and feature F Conv1_1 Multiply to get feature F channel1 ;

[0097] Step 2.8.5 Use the spatial_attention1 module to focus on feature F channel1 Performing a treatment, comprising the following steps (d1) to (f1);

[0098] (d1) Use the parallel AvgPool1 layer and MaxPool1 layer to process the feature F channel1 Pooling operation is performed along the third dimension to obtain features F avgpool1 and F maxpool1 ;

[0099] (e1) Use the custom connection layer Concat1 to connect F avgpool1 and F maxpool1 Get feature F Concat1 ;

[0100] (f1) Use the convolution layer Conv1_5 to transform the feature F Concat1 Calculate and get the weight coefficient F s_a1 ;

[0101] Step 2.8.6: Set the weight coefficient F s_a1 and feature F channel1 Multiply to get feature F spatial1 ;

[0102] Step 2.8.7 Use convolutional layer Conv1_2 to transform feature F spatial1 Perform feature extraction to obtain the encoding subnetwork N encx The output feature F x_code ;

[0103] Step 2.9 uses the encoding subnetwork N ency extract Features;

[0104] Step 2.9.1 Using the encoding subnetwork N ency The first layer of Wavelet2 Perform feature extraction to obtain texture information feature F Wavelet2 ;

[0105] Step 2.9.2 Use convolutional layers Conv2_0 and Conv2_1 to perform F Wavelet2 Perform feature extraction and obtain feature F Conv2_1 ;

[0106] Step 2.9.3 Use the channel_attention2 module to focus on the feature F Conv2_1 Performing a treatment, comprising the following steps (a2) to (c2);

[0107] (a2) Use the parallel GlobalAvgPool2 layer and GlobalMaxPool2 layer to perform feature Conv2_1 Perform pooling operations to obtain features F Globalavg2 and F Globalmax2 ;

[0108] (b2) Use convolutional layers Conv2_3 and Conv2_4 to successively process feature F Globalavg2 Perform feature extraction to obtain F Gavgout2 , using convolutional layers Conv2_3 and Conv2_4 to successively Globalmax2 Perform feature extraction to obtain F Gmaxout2 ;

[0109] (c2) for feature F Gavgout2 and F Gmaxout2 Perform a summation operation, and then use the nonlinear activation function Sigmoid to calculate the summation result to obtain the weight coefficient F c_a2 ;

[0110] Step 2.9.4: Set the weight coefficient F c_a2 and feature F Conv2_1 Multiply to get feature F channel2 ;

[0111] Step 2.9.5 Use the spatial_attention2 module to focus on feature F channel2 Performing a treatment, comprising the following steps (d2) to (f2);

[0112] (d2) Use the parallel AvgPool2 layer and MaxPool2 layer to process the feature F channel2 Pooling operation is performed along the third dimension to obtain features F avgpool2 and F maxpool2 ;

[0113] (e2) Use the custom connection layer Concat2 to connect F avgpool2 and F maxpool2 Get feature F Concat2 ;

[0114] (f2) Use the convolution layer Conv2_5 to perform feature F Concat2 Calculate and get the weight coefficient F s_a2 ;

[0115] Step 2.9.6: Set the weight coefficient F s_a2 and feature F channel2 Multiply to get feature F spatial2 ;

[0116] Step 2.9.7 Use the convolutional layer Conv2_2 to transform the feature F spatial2 Perform feature extraction to obtain subnetwork N ency The output feature F y_code ;

[0117] Step 2.10: Use subnetwork N decx For feature F x_codeRefactoring

[0118] Step 2.10.1 Using subnetwork N decx The first convolutional layer Conv3_0 is used for F x_code Perform feature extraction and obtain feature F Conv3_0 ;

[0119] Step 2.10.2 Use the channel_attention3 module to focus on the feature F Conv3_0 Performing a treatment, comprising the following steps (a3) ​​to (c3);

[0120] (a3) Use the parallel GlobalAvgPool3 layer and GlobalMaxPool3 layer to perform feature Conv3_0 Perform pooling operations to obtain features F Globalavg3 and F Globalmax3 ;

[0121] (b3) Use convolutional layers Conv3_3 and Conv3_4 to successively Globalavg3 Perform feature extraction to obtain F Gavgout3 , using convolutional layers Conv3_3 and Conv3_4 to successively Globalmax3 Perform feature extraction to obtain F Gmaxout3 ;

[0122] (c3) for feature F Gavgout3 and F Gmaxout3 Perform a summation operation, and then use the nonlinear activation function Sigmoid to calculate the summation result to obtain the weight coefficient F c_a3 ;

[0123] Step 2.10.3: Set the weight coefficient F c_a3 and feature F Conv3_0 Multiply to get feature F channel3 ;

[0124] Step 2.10.4 Use the spatial_attention3 module to focus on feature F channele Performing a treatment, comprising the following steps (d3) to (f3);

[0125] (d3) Use parallel AvgPool3 and MaxPool3 layers to process feature F channel3 Pooling operation is performed along the third dimension to obtain features F avgpool3 and F maxpool3 ;

[0126] (e3) Use the custom connection layer Concat3 to connect F avgpool3 and F maxpool3 Get feature F Concat3 ;

[0127] (f3) Use the convolution layer Conv3_5 to transform the feature F Concat3 Calculate and get the weight coefficient F s_a3 ;

[0128] Step 2.10.5: Set the weight coefficient F s_a3 and feature F channel3 Multiply to get feature F spatial3 ;

[0129] Step 2.10.6 Use convolutional layers Conv3_1 and Conv3_2 to perform F spatial3 Perform feature extraction and obtain feature F Conv3_2 ;

[0130] Step 2.10.7 Use the inverse wavelet layer Rewavelet1 to transform the feature F Conv3_2 Reconstruct and get sub-network N decx Output

[0131] Step 2.11 Use subnetwork N decy For feature F y_code Refactoring

[0132] Step 2.11.1 Using subnetwork N decy The first convolutional layer Conv4_0 is used for F y_code Perform feature extraction and obtain feature F Conv4_0 ;

[0133] Step 2.11.2 Use the channel_attention4 module to focus on the feature F Conv4_0 Performing a treatment, comprising the following steps (a4) to (c4);

[0134] (a4) Use the parallel GlobalAvgPool4 layer and GlobalMaxPool4 layer to perform feature Conv4_0 Perform pooling operations to obtain features F Globalavg4 and F Globalmax4 ;

[0135] (b4) Use convolutional layers Conv4_3 and Conv4_4 to successively Globalavg4 Perform feature extraction to obtain F Gavgout4 , using convolutional layers Conv4_3 and Conv4_4 to successively Globalmax4 Perform feature extraction to obtain F Gmaxout4 ;

[0136] (c4) for feature F Gavgout4 and F Gmaxout4Perform a summation operation, and then use the nonlinear activation function Sigmoid to calculate the summation result to obtain the weight coefficient F c_a4 ;

[0137] Step 2.11.3: Set the weight coefficient F c_a4 and feature F Conv4_0 Multiply to get feature F channel4 ;

[0138] Step 2.11.4 Use the spatial_attention4 module to focus on feature F channel4 Performing a treatment, comprising the following steps (d4) to (f4);

[0139] (d4) Use parallel AvgPool4 and MaxPool4 layers to process feature F channel4 Pooling operation is performed along the third dimension to obtain features F avgpool4 and F maxpool4 ;

[0140] (e4) Use the custom connection layer Concat4 to connect F avgpool4 and F maxpool4 Get feature F Concat4 ;

[0141] (f4) Use the convolution layer Conv4_5 to perform feature F Concat4 Calculate and get the weight coefficient F s_a4 ;

[0142] Step 2.11.5: Set the weight coefficient F s_a4 and feature F channel4 Multiply to get feature F spatial4 ;

[0143] Step 2.11.6 Use convolutional layers Conv4_1 and Conv4_2 to perform F spatial4 Perform feature extraction and obtain feature F Conv4_2 ;

[0144] Step 2.11.7 Use the inverse wavelet layer Rewavelet2 to transform the feature F Conv4_2 Reconstruct and get sub-network N decy Output

[0145] Step 2.12: Use subnetwork N decx For feature F y_code Reconstruction to obtain X′ H ;

[0146] Step 2.13 Use subnetwork N decy For feature F x_code Reconstruct and obtain Y′H ;

[0147] Step 2.14 Use subnetwork N encx X′ H Perform feature extraction to obtain F′ x_code , and then use sub-network N decy For feature F′ x_code Reconstruction to obtain

[0148] Step 2.15: Use subnetwork N ency Y′ H Perform feature extraction to obtain F′ y_code , and then use sub-network N decx For feature F′ y_code Reconstruction to obtain

[0149] Step 2.16 Based on the definitions of formulas (7)-(10), use X′ H and Y′ H Calculate the change detection result map Change_map;

[0150]

[0151]

[0152]

[0153] Change_map = Otsu (diff) (10)

[0154] Among them C X is an image The number of channels, C Y is an image The number of channels, Otsu(·) represents the Otsu method;

[0155] Step 2.17 According to the definition of formulas (11)-(15), use the loss function L enc Pair network N encx and N ency Update parameters;

[0156]

[0157]

[0158]

[0159]

[0160]

[0161] Where affinity(·) represents the affinity matrix of the image, C x_code Indicates F x_code The number of channels, M represents the number of pixels of S;

[0162] Step 2.18 According to the definitions of formulas (16)-(19), use the loss function L to calculate the loss of the entire network N hic Update parameters;

[0163]

[0164]

[0165]

[0166] L=α1L1+α2L2+α3L3 (19)

[0167] Where N represents The number of pixels, clw represents the change prior, α1, α2 and α3 represent the preset weight coefficients;

[0168] Step 2.19: If all pixel blocks in the training set have been processed, go to step 2.20; otherwise, take a group of unprocessed pixel blocks from the training set and return to step 2.8;

[0169] Step 2.20 Let iter←iter+1. If the number of iterations iter>Total_iter, the trained neural network N is obtained. hic , go to step 3; otherwise, use the Adam-based back-error propagation algorithm and the prediction loss L enc and L updates N hic , go to step 2.8 to reprocess all pixel blocks in the training set, and Total_iter represents the preset number of iterations.

[0170] The step 3 is as follows:

[0171] Step 3.1 Use the trained sub-network N encx extract Features;

[0172] Step 3.1.1 Using subnetwork N encx The first layer of Wavelet1 Perform feature extraction to obtain texture information feature P Wavelet1 ;

[0173] Step 3.1.2 Use convolutional layers Conv1_0 and Conv1_1 to sequentially Wavelet1 Perform feature extraction and obtain feature P Conv1_1 ;

[0174] Step 3.1.3 Use the channel_attention1 module to focus on the feature P Conv1_1 Processing includes the following steps a1-c1;

[0175] (a1) Use the parallel GlobalAvgPool1 layer and GlobalMaxPool1 layer to perform feature Conv1_1 Perform pooling operations to obtain features P Globalavg1 and P Globalmax1 ;

[0176] (b1) Use convolutional layers Conv1_3 and Conv1_4 to sequentially process feature P Globalavg1 Perform feature extraction to obtain P Gavgout1 , using convolutional layers Conv1_3 and Conv1_4 to successively Globalmax1 Perform feature extraction to obtain P Gmaxout1 ;

[0177] (c1) For feature P Gavgout1 and P Gmaxout1 Perform a summation operation, and then use the nonlinear activation function Sigmoid to calculate the summation result to obtain the weight coefficient P c_a1 ;

[0178] Step 3.1.4 Set the weight coefficient P c_a1 and feature P Conv1_1 Multiply to get feature P channel1 ;

[0179] Step 3.1.5 Use the spatial_attention1 module to focus on feature P channel1 Performing a treatment, comprising the following steps (d1) to (f1);

[0180] (d1) Use the parallel AvgPool1 layer and MaxPool1 layer to process the feature P channel1 Pooling operation is performed along the third dimension to obtain features P avgpool1 and P maxpool1 ;

[0181] (e1) Use the custom connection layer Concat1 to connect P avgpool1 and P maxpool1 Get feature P Concat1 ;

[0182] (f1) Use the convolution layer Conv1_5 to transform the feature PConcat1 Calculate and get the weight coefficient P s_a1 ;

[0183] Step 3.1.6 Set the weight coefficient P s_a1 and feature P channel1 Multiply to get feature P spatial1 ;

[0184] Step 3.1.7 Use convolutional layer Conv1_2 to transform feature P spatial1 Perform feature extraction to obtain subnetwork N encx The output feature P x_code ;

[0185] Step 3.2: Use the trained sub-network N ency extract Features;

[0186] Step 3.2.1 Using subnetwork N ency The first layer of Wavelet2 Perform feature extraction to obtain texture information feature P Wavelet2 ;

[0187] Step 3.2.2 Use convolutional layers Conv2_0 and Conv2_1 to sequentially Wavelet2 Perform feature extraction and obtain feature P Conv2_1 ;

[0188] Step 3.2.3 Use the channel_attention2 module to focus on the feature P Conv2_1 Processing includes the following steps a2-c2;

[0189] (a2) Use the parallel GlobalAvgPool2 layer and GlobalMaxPool2 layer to perform feature Conv2_1 Perform pooling operations to obtain features P Globalavg2 and P Globalmax2 ;

[0190] (b2) Use convolutional layers Conv2_3 and Conv2_4 to successively Globalavg2 Perform feature extraction to obtain P Gavgout2 , use convolutional layers Conv2_3 and Conv2_4 to successively Globalmax2 Perform feature extraction to obtain P Gmaxout2 ;

[0191] (c2) For feature P Gavgout2 and P Gmaxout2 Perform a summation operation, and then use the nonlinear activation function Sigmoid to calculate the summation result to obtain the weight coefficient P c_a2 ;

[0192] Step 3.2.4 Set the weight coefficient P c_a2 and feature P Conv2_1 Multiply to get feature P channel2 ;

[0193] Step 3.2.5 Use the spatial_attention2 module to focus on feature P channel2 Performing a treatment, comprising the following steps (d2) to (f2);

[0194] (d2) Use the parallel AvgPool2 layer and MaxPool2 layer to process the feature P channel2 Pooling operation is performed along the third dimension to obtain features P avgpool2 and P maxpool2 ;

[0195] (e2) Use the custom connection layer Concat2 to connect P avgpool2 and P maxpool2 Get feature P Concat2 ;

[0196] (f2) Use the convolution layer Conv2_5 to transform the feature P Concat2 Calculate and get the weight coefficient P s_a2 ;

[0197] Step 3.2.6: Set the weight coefficient P s_a2 and feature P channel2 Multiply to get feature P spatial2 ;

[0198] Step 3.2.7 Use the convolutional layer Conv2_2 to transform the feature P spatial2 Perform feature extraction to obtain subnetwork N ency The output feature P y_code ;

[0199] Step 3.3: Use the trained sub-network N decx For feature P x_code Refactoring

[0200] Step 3.3.1 Using subnetwork N decx The first convolutional layer Conv3_0 has a x_code Perform feature extraction and obtain feature P Conv3_0 ;

[0201] Step 3.3.2 Use the channel_attention3 module to focus on the feature P Conv3_0 Performing a treatment, comprising the following steps (a3) ​​to (c3);

[0202] (a3) Use the parallel GlobalAvgPool3 layer and GlobalMaxPool3 layer to perform feature Conv3_0 Perform pooling operations to obtain features P Globalavg3 and P Globalmax3 ;

[0203] (b3) Use convolutional layers Conv3_3 and Conv3_4 to successively process feature P Globalavg3 Perform feature extraction to obtain P Gavgout3 , using convolutional layers Conv3_3 and Conv3_4 to successively Globalmax3 Perform feature extraction to obtain P Gmaxout3 ;

[0204] (c3) for feature F Gavgout3 and P Gmaxout3 Perform a summation operation, and then use the nonlinear activation function Sigmoid to calculate the summation result to obtain the weight coefficient P c_a3 ;

[0205] Step 3.3.3 Set the weight coefficient P c_a3 and feature P Conv3_0 Multiply to get feature P channel3 ;

[0206] Step 3.3.4 Use the spatial_attention3 module to focus on feature P channel3 Performing a treatment, comprising the following steps (d3) to (f3);

[0207] (d3) Use the parallel AvgPool3 layer and MaxPool3 layer to process the feature P channel3 Pooling operation is performed along the third dimension to obtain features P avgpool3 and P maxpool3 ;

[0208] (e3) Use the custom connection layer Concat3 to connect P avgpool3 and P maxpool3 Get feature P Concat3 ;

[0209] (f3) Use the convolution layer Conv3_5 to transform the feature P Concat3 Calculate and get the weight coefficient P s_a3 ;

[0210] Step 3.3.5: Set the weight coefficient P s_a3 and feature P channel3 Multiply to get feature P spatial3 ;

[0211] Step 3.3.6 Use convolutional layers Conv3_1 and Conv3_2 to sequentiallyspatial3 Perform feature extraction and obtain feature P Conv3_2 ;

[0212] Step 3.3.7 Use the inverse wavelet layer Rewavelet1 to transform the feature P Conv3_2 Reconstruct and get sub-network N decx Output

[0213] Step 3.4: Use the trained sub-network N decy For feature P y_code Refactoring

[0214] Step 3.4.1 Using subnetwork N decy The first convolutional layer Conv4_0 has a y_code Perform feature extraction and obtain feature P Conv4_0 ;

[0215] Step 3.4.2 Use the channel_attention4 module to focus on the feature P Conv4_0 Performing a treatment, comprising the following steps (a4) to (c4);

[0216] (a4) Use the parallel GlobalAvgPool4 layer and GlobalMaxPool4 layer to perform feature Conv4_0 Perform pooling operations to obtain features P Globalavg4 and P Globalmax4 ;

[0217] (b4) Use convolutional layers Conv4_3 and Conv4_4 to successively Globalavg4 Perform feature extraction to obtain P Gavgout4 , using convolutional layers Conv4_3 and Conv4_4 to successively Globalmax4 Perform feature extraction to obtain P Gmaxout4 ;

[0218] (c4) For feature P Gavgout4 and P Gmaxout4 Perform a summation operation, and then use the nonlinear activation function Sigmoid to calculate the summation result to obtain the weight coefficient P c_a4 ;

[0219] Step 3.4.3 Set the weight coefficient P c_a4 and feature P Conv4_0 Multiply to get feature P channel4 ;

[0220] Step 3.4.4 Use the spatial_attention4 module to focus on feature P channel4 Performing a treatment, comprising the following steps (d4) to (f4);

[0221] (d4) Use the parallel AvgPool4 layer and MaxPool4 layer to process the feature P channel4 Pooling operation is performed along the third dimension to obtain features P avgpool4 and P maxpool4 ;

[0222] (e4) Use the custom connection layer Concat4 to connect P avgpool4 and P maxpool4 Get feature P Concat4 ;

[0223] (f4) Use the convolution layer Conv4_5 to transform the feature P Concat4 Calculate and get the weight coefficient P s_a4 ;

[0224] Step 3.4.5: Set the weight coefficient P s_a4 and feature P channel4 Multiply to get feature P spatial4 ;

[0225] Step 3.4.6 Use convolutional layers Conv4_1 and Conv4_2 to sequentially spatial4 Perform feature extraction and obtain feature P Conv4_2 ;

[0226] Step 3.4.7 Use the inverse wavelet layer Rewavelet2 to transform the feature P Conv4_2 Reconstruct and get sub-network N decy Output

[0227] Step 3.5: Use subnetwork N decx For feature P y_code Reconstruction to obtain X′ P ;

[0228] Step 3.6 Use subnetwork N decy For feature P x_code Reconstruct and obtain Y′ P ;

[0229] Step 3.7 Based on the definitions of formulas (20)-(23), use X′ P and Y′ P Calculate the change detection result map Change_map1;

[0230]

[0231]

[0232]

[0233] Change_map1=Otsu(diff1) (23)

[0234] Among them C X1 is an image The number of channels, C Y1 is an image The number of channels, Otsu(·) represents the Otsu method.

[0235] Compared with the prior art, the present invention has two advantages: first, a neural network framework for heterogeneous remote sensing image change detection based on topological structure coupling is designed, which can effectively realize the conversion of image domains, thereby calculating the difference map in the same domain, and combining the difference maps generated in two different domains for final change detection, which can effectively improve the accuracy of change detection. Second, adding a wavelet layer to the network can effectively capture the texture structure features of the input image, highlight the structural information of the original image, and then allocate more weights to the region of interest by introducing the channel attention mechanism and the spatial attention mechanism. Through the organic combination of wavelet and channel attention mechanism and spatial attention mechanism, the influence of the significantly different pixel values ​​of heterogeneous images on the network can be greatly reduced, so that the network can focus on the texture structure information of interest, thereby suppressing unnecessary features and greatly improving the accuracy of heterogeneous remote sensing image change detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0236] Figure 1 The change detection network N in the embodiment of the present invention hic Structure and flow chart.

[0237] Figure 2 This is the channel_attention1 module structure and flow chart of an embodiment of the present invention.

[0238] Figure 3 This is the structure and flow chart of the spatial_attention1 module of an embodiment of the present invention.

[0239] Figure 4 The figure is a schematic diagram comparing the change detection results of the Texas data set between the embodiment of the present invention and the ACE-Net method, the CAN method, the SCCN method, the NPSG method, and the INLPG method.

[0240] Figure 5 The figure is a schematic diagram comparing the change detection results of the California data set using the embodiment of the present invention, the ACE-Net method, the CAN method, the SCCN method, the NPSG method, and the INLPG method.

[0241] Figure 6It is a schematic diagram comparing the change detection results of the Sardinia data set between the embodiment of the present invention and the ACE-Net method, the CAN method, the SCCN method, the NPSG method, and the INLPG method. DETAILED DESCRIPTION

[0242] A heterogeneous remote sensing image change detection method based on topological structure coupling of the present invention is as follows Figure 1 As shown, proceed as follows:

[0243] Step 1. Establish and initialize the convolutional neural network N for heterogeneous remote sensing image change detection hic , the N hic Contains 2 encoder sub-networks N encx and N ency And 2 decoder sub-networks N decx and N decy ;

[0244] Step 1.1 Create and initialize subnetwork N encx , the sub-network N encx Contains 1 wavelet layer Wavelet1, 3 convolutional layers, namely Conv1_0, Conv1_1 and Conv1_2, as well as 1 channel_attention1 module and 1 spatial_attention1 module;

[0245] The Wavelet1 includes 1 layer of wavelet operation, 1 layer of activation operation and 1 layer of Dropout operation, wherein the wavelet layer uses a two-dimensional Haar wavelet transform to decompose the image to form 1 low-frequency image and 3 high-frequency images, and selects a nonlinear activation function LeakyReLU with a parameter of 0.3 as the activation function and a Dropout operation with a parameter of 0.2 for operation;

[0246] Conv1_0 includes 1 convolution operation, 1 activation operation and 1 Dropout operation, wherein the convolution layer contains 50 convolution kernels of size 3×3, each convolution kernel performs convolution operation with a step size of 1 pixel, and uses the nonlinear activation function LeakyReLU with a parameter of 0.3 as the activation function and the Dropout operation with a parameter of 0.2 for operation;

[0247] The Conv1_1 includes 1 convolution operation, 1 activation operation and 1 Dropout operation, wherein the convolution layer contains 50 convolution kernels of size 3×3, each convolution kernel performs convolution operation with a step size of 1 pixel, and uses the nonlinear activation function LeakyReLU with a parameter of 0.3 as the activation function and the Dropout operation with a parameter of 0.2 for operation;

[0248] Conv1_2 includes 1 convolution operation and 1 activation operation, wherein the convolution layer contains 3 convolution kernels of size 3×3, each convolution kernel performs convolution operation with a step size of 1 pixel, and selects nonlinear activation function Tanh as activation function for operation;

[0249] The channel_attention1 module is as follows Figure 2 As shown: it contains 1 group of GlobalAvgPool1 layers, 1 group of GlobalMaxPool1 layers, and 2 groups of convolutional layers, namely Conv1_3 and Conv1_4, and the nonlinear activation function Sigmoid is used to calculate the summation results;

[0250] The GlobalAvgPool1 includes 1 layer of GlobalAveragePooling2D operation and 1 layer of Reshape operation;

[0251] The GlobalMaxPool1 includes 1 layer of GlobalMaxPooling2D operation and 1 layer of Reshape operation;

[0252] Conv1_3 includes 1 convolution operation and 1 activation operation, wherein the convolution layer contains 4 convolution kernels of size 1×1, each convolution kernel performs convolution operation with a step size of 1 pixel, and uses the nonlinear activation function Relu as the activation function for operation;

[0253] Conv1_4 includes one convolution operation, wherein the convolution layer contains 50 convolution kernels of size 1×1, and each convolution kernel performs convolution operation with a step size of 1 pixel;

[0254] The spatial_attention1 is as Figure 3 As shown: it contains 1 AvgPool1 layer, 1 MaxPool1 layer, a custom connection layer Concat1, and 1 convolution layer Conv1_5;

[0255] AvgPool1 contains one reduce_mean operation, which performs calculation along the third dimension;

[0256] The MaxPool1 contains one reduce_max operation, which performs calculation along the third dimension;

[0257] The custom connection layer Concat1 is used to connect two features;

[0258] Conv1_5 includes one convolution operation and one activation operation, wherein the convolution layer contains one convolution kernel of size 7×7, each convolution kernel performs convolution operation with a step size of 1 pixel, and uses nonlinear activation function Sigmoid as activation function for operation;

[0259] Step 1.2. Create and initialize subnetwork N ency , the sub-network N ency Contains 1 wavelet layer Wavelet2, 3 convolutional layers, namely Conv2_0, Conv2_1 and Conv2_2, 1 channel_attention2 module and 1 spatial_atention2 module;

[0260] The Wavelet2 includes 1 layer of wavelet operation, 1 layer of activation operation and 1 layer of Dropout operation, wherein the wavelet layer uses a two-dimensional Haar wavelet transform to decompose the image to form 1 low-frequency image and 3 high-frequency images, and selects a nonlinear activation function LeakyReLU with a parameter of 0.3 as the activation function and a Dropout operation with a parameter of 0.2 for operation;

[0261] The Conv2_0 includes 1 convolution operation, 1 activation operation and 1 Dropout operation, wherein the convolution layer contains 50 convolution kernels of size 3×3, each convolution kernel performs convolution operation with a step size of 1 pixel, and uses the nonlinear activation function LeakyReLU with a parameter of 0.3 as the activation function and the Dropout operation with a parameter of 0.2 for operation;

[0262] The Conv2_1 includes 1 convolution operation, 1 activation operation and 1 Dropout operation, wherein the convolution layer contains 50 convolution kernels of size 3×3, each convolution kernel performs convolution operation with a step size of 1 pixel, and uses the nonlinear activation function LeakyReLU with a parameter of 0.3 as the activation function and the Dropout operation with a parameter of 0.2 for operation;

[0263] The Conv2_2 includes 1 convolution operation and 1 activation operation, wherein the convolution layer contains 3 convolution kernels of size 3×3, each convolution kernel performs convolution operation with a step size of 1 pixel, and selects the nonlinear activation function Tanh as the activation function for operation;

[0264] The channel_attention2 module includes 1 set of GlobalAvgPool2 layers, 1 set of GlobalMaxPool2 layers, and 2 sets of convolutional layers, namely Conv2_3 and Conv2_4, and uses the nonlinear activation function Sigmoid to calculate the summation result;

[0265] The GlobalAvgPool2 includes 1 layer of GlobalAveragePooling2D operation and 1 layer of Reshape operation;

[0266] The GlobalMaxPool2 includes 1 layer of GlobalMaxPooling2D operation and 1 layer of Reshape operation;

[0267] Conv2_3 includes 1 convolution operation and 1 activation operation, wherein the convolution layer contains 4 convolution kernels of size 1×1, each convolution kernel performs convolution operation with a step size of 1 pixel, and uses the nonlinear activation function Relu as the activation function for operation;

[0268] Conv2_4 includes one convolution operation, wherein the convolution layer contains 50 convolution kernels of size 1×1, and each convolution kernel performs convolution operation with a step size of 1 pixel;

[0269] The spatial_attention2 module includes 1 AvgPool2 layer, 1 MaxPool2 layer, a custom connection layer Concat2 and 1 convolution layer Conv2_5;

[0270] The AvgPool2 contains 1 reduce_mean operation, which performs calculation along the third dimension;

[0271] The MaxPool2 contains 1 reduce_max operation, which performs calculation along the third dimension;

[0272] The custom connection layer Concat2 is used to connect two features;

[0273] Conv2_5 includes one convolution operation and one activation operation, wherein the convolution layer contains one convolution kernel of size 7×7, each convolution kernel performs convolution operation with a step size of 1 pixel, and uses nonlinear activation function Sigmoid as activation function for operation;

[0274] Step 1.3. Create and initialize subnetwork N decx , the sub-network N decx Contains 3 groups of convolutional layers, namely Conv3_0, Conv3_1 and Conv3_2, 1 group of channel_attention3 modules and 1 group of spatial_attention3 modules and 1 group of inverse wavelet layer Rewavelet1;

[0275] The Conv3_0 includes 1 convolution operation, 1 activation operation and 1 Dropout operation, wherein the convolution layer contains 50 convolution kernels of size 3×3, each convolution kernel performs convolution operation with a step size of 1 pixel, and uses the nonlinear activation function LeakyReLU with a parameter of 0.3 as the activation function and the Dropout operation with a parameter of 0.2 for operation;

[0276] The Conv3_1 includes 1 convolution operation, 1 activation operation and 1 Dropout operation, wherein the convolution layer contains 50 convolution kernels of size 3×3, each convolution kernel performs convolution operation with a step size of 1 pixel, and uses the nonlinear activation function LeakyReLU with a parameter of 0.3 as the activation function and the Dropout operation with a parameter of 0.2 for operation;

[0277] Conv3_2 contains 1 convolution operation, 1 activation operation and 1 Dropout operation, where the convolution layer contains C x ×4 convolution kernels of size 3×3, where C x is the number of channels of the input image, each convolution kernel performs convolution operation with a step size of 1 pixel, and uses the nonlinear activation function LeakyReLU with a parameter of 0.3 as the activation function and the Dropout operation with a parameter of 0.2 for operation;

[0278] The Rewavelet1 includes one layer of inverse wavelet operation and one layer of activation operation, wherein the inverse wavelet layer uses two-dimensional Haar wavelet inverse transform to reconstruct the image, and uses nonlinear activation function Tanh as activation function for operation;

[0279] The channel_attention3 module includes 1 group of GlobalAvgPool3 layers, 1 group of GlobalMaxPool3 layers, and 2 groups of convolutional layers, namely Conv3_3 and Conv3_4, and uses the nonlinear activation function Sigmoid to calculate the summation result;

[0280] The GlobalAvgPool3 includes 1 layer of GlobalAveragePooling2D operation and 1 layer of Reshape operation;

[0281] The GlobalMaxPool3 includes 1 layer of GlobalMaxPooling2D operation and 1 layer of Reshape operation;

[0282] Conv3_3 includes 1 convolution operation and 1 activation operation, wherein the convolution layer contains 4 convolution kernels of size 1×1, each convolution kernel performs convolution operation with a step size of 1 pixel, and uses the nonlinear activation function Relu as the activation function for operation;

[0283] Conv3_4 includes one convolution operation, wherein the convolution layer contains 50 convolution kernels of size 1×1, and each convolution kernel performs convolution operation with a step size of 1 pixel;

[0284] The spatial_attention3 module includes 1 AvgPool3 layer, 1 MaxPool3 layer, a custom connection layer Concat3 and 1 convolution layer Conv3_5;

[0285] The AvgPool3 contains 1 reduce_mean operation, which performs calculation along the third dimension;

[0286] The MaxPool3 contains 1 reduce_max operation, which performs calculation along the third dimension;

[0287] The custom connection layer Concat3 is used to connect two features;

[0288] Conv3_5 includes one convolution operation and one activation operation, wherein the convolution layer contains one convolution kernel of size 7×7, each convolution kernel performs convolution operation with a step size of 1 pixel, and uses the nonlinear activation function Sigmoid as the activation function for operation;

[0289] Step 1.4. Create and initialize subnetwork N decy , the sub-network N decy Contains 3 groups of convolutional layers, namely Conv4_0, Conv4_1 and Conv4_2, 1 group of channel_attention4 modules and 1 group of spatial_attention4 modules and 1 group of inverse wavelet layer Rewavelet2;

[0290] The Conv4_0 includes 1 convolution operation, 1 activation operation and 1 Dropout operation, wherein the convolution layer contains 50 convolution kernels of size 3×3, each convolution kernel performs convolution operation with a step size of 1 pixel, and uses the nonlinear activation function LeakyReLU with a parameter of 0.3 as the activation function and the Dropout operation with a parameter of 0.2 for operation;

[0291] The Conv4_1 includes 1 convolution operation, 1 activation operation and 1 Dropout operation, wherein the convolution layer contains 50 convolution kernels of size 3×3, each convolution kernel performs convolution operation with a step size of 1 pixel, and uses the nonlinear activation function LeakyReLU with a parameter of 0.3 as the activation function and the Dropout operation with a parameter of 0.2 for operation;

[0292] Conv4_2 includes 1 convolution operation, 1 activation operation and 1 Dropout operation, where the convolution layer contains C y ×4 convolution kernels of size 3×3, where C y is the number of channels of the input image, each convolution kernel performs convolution operation with a step size of 1 pixel, and uses the nonlinear activation function LeakyReLU with a parameter of 0.3 as the activation function and the Dropout operation with a parameter of 0.2 for operation;

[0293] The Rewavelet2 includes one layer of inverse wavelet operation and one layer of activation operation, wherein the inverse wavelet layer uses two-dimensional Haar wavelet inverse transform to reconstruct the image, and uses nonlinear activation function Tanh as activation function for operation;

[0294] The channel_attention4 module includes 1 group of GlobalAvgPool4 layers, 1 group of GlobalMaxPool4 layers, and 2 groups of convolutional layers, namely Conv4_3 and Conv4_4, and uses the nonlinear activation function Sigmoid to calculate the summation result;

[0295] The GlobalAvgPool4 includes 1 layer of GlobalAveragePooling2D operation and 1 layer of Reshape operation;

[0296] The GlobalMaxPool4 includes 1 layer of GlobalMaxPooling2D operation and 1 layer of Reshape operation;

[0297] The Conv4_3 includes 1 convolution operation and 1 activation operation, wherein the convolution layer contains 4 convolution kernels of size 1×1, each convolution kernel performs convolution operation with a step size of 1 pixel, and uses the nonlinear activation function Relu as the activation function for operation;

[0298] Conv4_4 includes one convolution operation, wherein the convolution layer contains 50 convolution kernels of size 1×1, and each convolution kernel performs convolution operation with a step size of 1 pixel;

[0299] The spatial_attention4 module includes 1 AvgPool4 layer, 1 MaxPool4 layer, a custom connection layer Concat4 and 1 convolution layer Conv4_5;

[0300] The AvgPool4 contains 1 reduce_mean operation, which is calculated along the third dimension;

[0301] The MaxPool4 contains 1 reduce_max operation, which performs calculation along the third dimension;

[0302] The custom connection layer Concat4 is used to connect two features;

[0303] Conv4_5 includes one convolution operation and one activation operation, wherein the convolution layer contains one convolution kernel of size 7×7, each convolution kernel performs convolution operation with a step size of 1 pixel, and uses the nonlinear activation function Sigmoid as the activation function for operation;

[0304] Step 2: Input the training set H of heterogeneous remote sensing images and train N hic Perform unsupervised training;

[0305] Step 2.1. Standardize the input remote sensing image X according to formula (1), (2), (3) to obtain Where X = {X i |i=1,...,C1},X i represents the i-th channel of X, represent The i-th channel of , C1 represents the number of channels of X;

[0306] A=mean(X i )+3×std(X i ) (1)

[0307]

[0308]

[0309] Among them, mean(·) represents the mean of the image, std(·) represents the standard deviation of the image, and max(·) represents the maximum value of the image;

[0310] Step 2.2. According to formula (4), (5), (6), the input remote sensing image Y is standardized to obtain Where Y = {Y j |j=1,...,C2},Y j represents the j-th channel of Y, represent The jth channel of , C2 represents the number of channels of Y;

[0311] B=mean(Y j )+3×std(Y j ) (4)

[0312]

[0313]

[0314] Among them, mean(·) represents the mean of the image, std(·) represents the standard deviation of the image, and max(·) represents the maximum value of the image;

[0315] Step 2.3. and And extract pixel point sets respectively and in express The a-th pixel point in express The ath pixel point in , m represents the total number of selected pixel points;

[0316] Step 2.4. Each pixel point is centered on Divide into a series of pixel blocks X of size 100×100 H1 ,by Each pixel point is centered on Divide into a series of pixel blocks Y of size 100×100 H1 ;

[0317] Step 2.5. X H1 and Y H1 Each pixel block in is randomly rotated z×90 degrees counterclockwise with its center point as the rotation center to obtain the pixel block set X H2 and Y H2 , where z represents a value randomly selected from the set {1, 2, 3, 4};

[0318] Step 2.6. X H1 and Y H1 Each pixel block in is flipped upside down to obtain the pixel block set X H3 and Y H3 ;

[0319] Step 2.7. Order Will and As the training set of the change detection neural network, it is sent to the network for unsupervised training, where and represents the pixel block in the training set, set the number of iterations iter←1, and execute steps 2.8 to 2.19;

[0320] Step 2.8 Use subnetwork N encx extract Features;

[0321] Step 2.8.1 Using subnetwork N encx The first layer of Wavelet1 Perform feature extraction to obtain texture information feature F Wavelet1 ;

[0322] Step 2.8.2 Use convolutional layers Conv1_0 and Conv1_1 to perform F Wavelet1 Perform feature extraction and obtain feature F Conv1_1 ;

[0323] Step 2.8.3 Use the channel_attention1 module to focus on the feature F Conv1_1 Performing a treatment, comprising the following steps (a1) to (c1);

[0324] (a1) Use the parallel GlobalAvgPool1 layer and GlobalMaxPool1 layer to perform feature Conv1_1 Perform pooling operations to obtain features F Globalavg1 and F Globalmax1 ;

[0325] (b1) Use convolutional layers Conv1_3 and Conv1_4 to sequentially process feature F Globalavg1 Perform feature extraction to obtain F Gavgout1 , using convolutional layers Conv1_3 and Conv1_4 to successively Globalmax1 Perform feature extraction to obtain F Gmaxout1 ;

[0326] (c1) for feature F Gavgout1 and F Gmaxout1 Perform a summation operation, and then use the nonlinear activation function Sigmoid to calculate the summation result to obtain the weight coefficient F c_a1 ;

[0327] Step 2.8.4: Set the weight coefficient F c_a1 and feature F Conv1_1 Multiply to get feature F channel1 ;

[0328] Step 2.8.5 Use the spatial_attention1 module to focus on feature F channel1Performing a treatment, comprising the following steps (d1) to (f1);

[0329] (d1) Use the parallel AvgPool1 layer and MaxPool1 layer to process the feature F channel1 Pooling operation is performed along the third dimension to obtain features F avgpool1 and F maxpool1 ;

[0330] (e1) Use the custom connection layer Concat1 to connect F avgpool1 and F maxpool1 Get feature F Concat1 ;

[0331] (f1) Use the convolution layer Conv1_5 to transform the feature F Concat1 Calculate and get the weight coefficient F s_a1 ;

[0332] Step 2.8.6: Set the weight coefficient F s_a1 and feature F channel1 Multiply to get feature F spatial1 ;

[0333] Step 2.8.7 Use convolutional layer Conv1_2 to transform feature F spatial1 Perform feature extraction to obtain subnetwork N encx The output feature F x_code ;

[0334] Step 2.9: Use subnetwork N ency extract Features;

[0335] Step 2.9.1 Using subnetwork N ency The first layer of Wavelet2 Perform feature extraction to obtain texture information feature F Wavelet2 ;

[0336] Step 2.9.2 Use convolutional layers Conv2_0 and Conv2_1 to perform F Wavelet2 Perform feature extraction and obtain feature F Conv2_1 ;

[0337] Step 2.9.3 Use the channel_attention2 module to focus on the feature F Conv2_1 Performing a treatment, comprising the following steps (a2) to (c2);

[0338] (a2) Use the parallel GlobalAvgPool2 layer and GlobalMaxPool2 layer to perform feature Conv2_1 Perform pooling operations to obtain features F Globalavg2 and FGlobalmax2 ;

[0339] (b2) Use convolutional layers Conv2_3 and Conv2_4 to successively process feature F Globalavg2 Perform feature extraction to obtain F Gavgout2 , using convolutional layers Conv2_3 and Conv2_4 to successively Globalmax2 Perform feature extraction to obtain F Gmaxout2 ;

[0340] (c2) for feature F Gavgout2 and F Gmaxout2 Perform a summation operation, and then use the nonlinear activation function Sigmoid to calculate the summation result to obtain the weight coefficient F c_a2 ;

[0341] Step 2.9.4: Set the weight coefficient F c_a2 and feature F Conv2_1 Multiply to get feature F channel2 ;

[0342] Step 2.9.5 Use the spatial_attention2 module to focus on feature F channel2 Processing includes the following steps d2-f2;

[0343] (d2) Use the parallel AvgPool2 layer and MaxPool2 layer to process the feature F channel2 Pooling operation is performed along the third dimension to obtain features F avgpool2 and F maxpool2 ;

[0344] (e2) Use the custom connection layer Concat2 to connect F avgpool2 and F maxpool2 Get feature F Concat2 ;

[0345] (f2) Use the convolution layer Conv2_5 to perform feature F Concat2 Calculate and get the weight coefficient F s_a2 ;

[0346] Step 2.9.6: Set the weight coefficient F s_a2 and feature F channel2 Multiply to get feature F spatial2 ;

[0347] Step 2.9.7 Use the convolutional layer Conv2_2 to transform the feature F spatial2 Perform feature extraction to obtain subnetwork N ency The output feature F y_code ;

[0348] Step 2.10: Use subnetwork Ndecx For feature F x_code Refactoring

[0349] Step 2.10.1 Using subnetwork N decx The first convolutional layer Conv3_0 is used for F x_code Perform feature extraction and obtain feature F Conv3_0 ;

[0350] Step 2.10.2 Use the channel_attention3 module to focus on the feature F Conv3_0 Performing a treatment, comprising the following steps (a3) ​​to (c3);

[0351] (a3) Use the parallel GlobalAvgPool3 layer and GlobalMaxPool3 layer to perform feature Conv3_0 Perform pooling operations to obtain features F Globalavg3 and F Globalmax3 ;

[0352] (b3) Use convolutional layers Conv3_3 and Conv3_4 to successively Globalavg3 Perform feature extraction to obtain F Gavgout3 , using convolutional layers Conv3_3 and Conv3_4 to successively Globalmax3 Perform feature extraction to obtain F Gmaxout3 ;

[0353] (c3) for feature F Gavgout3 and F Gmaxout3 Perform a summation operation, and then use the nonlinear activation function Sigmoid to calculate the summation result to obtain the weight coefficient F c_a3 ;

[0354] Step 2.10.3: Set the weight coefficient F c_a3 and feature F Conv3_0 Multiply to get feature F channel3 ;

[0355] Step 2.10.4 Use the spatial_attention3 module to focus on feature F channel3 Performing a treatment, comprising the following steps (d3) to (f3);

[0356] (d3) Use parallel AvgPool3 and MaxPool3 layers to process feature F channel3 Pooling operation is performed along the third dimension to obtain features F avgpool3 and F maxpool3 ;

[0357] (e3) Use the custom connection layer Concat3 to connect F avgpool3 and Fmaxpool3 Get feature F Concat3 ;

[0358] (f3) Use the convolution layer Conv3_5 to transform the feature F Concat3 Calculate and get the weight coefficient F s_a3 ;

[0359] Step 2.10.5: Set the weight coefficient F s_a3 and feature F channel3 Multiply to get feature F spatial3 ;

[0360] Step 2.10.6 Use convolutional layers Conv3_1 and Conv3_2 to perform F spatial3 Perform feature extraction and obtain feature F Conv3_2 ;

[0361] Step 2.10.7 Use the inverse wavelet layer Rewavelet1 to transform the feature F Conv3_2 Reconstruct and get sub-network N decx Output

[0362] Step 2.11 Use subnetwork N decy For feature F y_code Refactoring

[0363] Step 2.11.1 Using subnetwork N decy The first convolutional layer Conv4_0 is used for F y_code Perform feature extraction and obtain feature F Conv4_0 ;

[0364] Step 2.11.2 Use the channel_attention4 module to focus on the feature F Conv4_0 Performing a treatment, comprising the following steps (a4) to (c4);

[0365] (a4) Use the parallel GlobalAvgPool4 layer and GlobalMaxPool4 layer to perform feature Conv4_0 Perform pooling operations to obtain features F Globalavg4 and F Globalmax4 ;

[0366] (b4) Use convolutional layers Conv4_3 and Conv4_4 to successively Globalavg4 Perform feature extraction to obtain F Gavgout4 , using convolutional layers Conv4_3 and Conv4_4 to successively Globalmax4 Perform feature extraction to obtain F Gmaxout4 ;

[0367] (c4) for feature FGavgout4 and F Gmaxout4 Perform a summation operation, and then use the nonlinear activation function Sigmoid to calculate the summation result to obtain the weight coefficient F c_a4 ;

[0368] Step 2.11.3: Set the weight coefficient F c_a4 and feature F Conv4_0 Multiply to get feature F channel4 ;

[0369] Step 2.11.4 Use the spatial_attention4 module to focus on feature F channel4 Performing a treatment, comprising the following steps (d4) to (f4);

[0370] (d4) Use parallel AvgPool4 and MaxPool4 layers to process feature F channel4 Pooling operation is performed along the third dimension to obtain features F avgpool4 and F maxpool4 ;

[0371] (e4) Use the custom connection layer Concat4 to connect F avgpool4 and F maxpool4 Get feature F Concat4 ;

[0372] (f4) Use the convolution layer Conv4_5 to perform feature F Concat4 Calculate and get the weight coefficient F s_a4 ;

[0373] Step 2.11.5: Set the weight coefficient F s_a4 and feature F channel4 Multiply to get feature F spatial4 ;

[0374] Step 2.11.6 Use convolutional layers Conv4_1 and Conv4_2 to perform F spatial4 Perform feature extraction and obtain feature F Conv4_2 ;

[0375] Step 2.11.7 Use the inverse wavelet layer Rewavelet2 to transform the feature F Conv4_2 Reconstruct and get sub-network N decy Output

[0376] Step 2.12: Use subnetwork N decx For feature F y_code Reconstruction to obtain X′ H ;

[0377] Step 2.13 Use subnetwork N decyFor feature F x_code Reconstruct and obtain Y′ H ;

[0378] Step 2.14 Use subnetwork N encx X′ H Perform feature extraction to obtain F′ x_code , and then use sub-network N decy For feature F′ x_code Reconstruction to obtain

[0379] Step 2.15: Use subnetwork N ency Y′ H Perform feature extraction to obtain F′ y_code , and then use sub-network N decx For feature F′ y_code Reconstruction to obtain

[0380] Step 2.16 Based on the definitions of formulas (7)-(10), use X′ H and Y′ H Calculate the change detection result map Change_map;

[0381]

[0382]

[0383]

[0384] Change_map = Otsu (diff) (10)

[0385] Among them C X is an image The number of channels, C Y is an image The number of channels, Otsu(·) represents the Otsu method;

[0386] Step 2.17 According to the definition of formulas (11)-(15), use the loss function L enc Pair network N encx and N ency Update parameters;

[0387]

[0388]

[0389]

[0390]

[0391]

[0392] Where affinity(·) represents the affinity matrix of the image, C x_code Indicates F x_code The number of channels, M represents the number of pixels of S;

[0393] Step 2.18 According to the definitions of formulas (16)-(19), use the loss function L to calculate the loss of the entire network N hic Update parameters;

[0394]

[0395]

[0396]

[0397] L=α1L1+α2L2+α3L3 (19)

[0398] Where N represents The number of pixels, clw represents the change prior, α1, α2 and α3 represent the preset weight coefficients;

[0399] Step 2.19: If all pixel blocks in the training set have been processed, go to step 2.20; otherwise, take a group of unprocessed pixel blocks from the training set and return to step 2.8;

[0400] Step 2.20 Let iter←iter+1. If the number of iterations iter>Total_iter, the trained neural network N is obtained. hic , go to step 3; otherwise, use the Adam-based back-error propagation algorithm and the prediction loss L enc and L updates N hic , go to step 2.8 to reprocess all pixel blocks in the training set, and the Total_iter represents the preset number of iterations;

[0401] Step 3. Input standardized heterogeneous remote sensing images and And use the trained convolutional neural network N hic Complete change detection;

[0402] Step 3.1 Use subnetwork N encx extract Features;

[0403] Step 3.1.1 Utilizing the subnetwork Nencx The first layer of Wavelet1 Perform feature extraction to obtain texture information feature P Wavelet1 ;

[0404] Step 3.1.2 Use convolutional layers Conv1_0 and Conv1_1 to sequentially Wavelet1 Perform feature extraction and obtain feature P Conv1_1 ;

[0405] Step 3.1.3 Use the channel_attention1 module to focus on the feature P Conv1_1 Processing includes the following steps a1-c1;

[0406] (a1) Use the parallel GlobalAvgPool1 layer and GlobalMaxPool1 layer to perform feature Conv1_1 Perform pooling operations to obtain features P Globalavg1 and P Globalmax1 ;

[0407] (b1) Use convolutional layers Conv1_3 and Conv1_4 to sequentially process feature P Globalavg1 Perform feature extraction to obtain P Gavgout1 , using convolutional layers Conv1_3 and Conv1_4 to successively Globalmax1 Perform feature extraction to obtain P Gmaxout1 ;

[0408] (c1) For feature P Gavgout1 and P Gmaxout1 Perform a summation operation, and then use the nonlinear activation function Sigmoid to calculate the summation result to obtain the weight coefficient P c_a1 ;

[0409] Step 3.1.4 Set the weight coefficient P c_a1 and feature P Conv1_1 Multiply to get feature P channel1 ;

[0410] Step 3.1.5 Use the spatial_attention1 module to focus on feature P channel1 Performing a treatment, comprising the following steps (d1) to (f1);

[0411] (d1) Use the parallel AvgPool1 layer and MaxPool1 layer to process the feature P channel1 Pooling operation is performed along the third dimension to obtain features P avgpool1 and P maxpool1 ;

[0412] (e1) Use the custom connection layer Concat1 to connect P avgpool1 and P maxpool1Get feature P Concat1 ;

[0413] (f1) Use the convolution layer Conv1_5 to transform the feature P Concat1 Calculate and get the weight coefficient P s_a1 ;

[0414] Step 3.1.6 Set the weight coefficient P s_a1 and feature P channel1 Multiply to get feature P spatial1 ;

[0415] Step 3.1.7 Use convolutional layer Conv1_2 to transform feature P spatial1 Perform feature extraction to obtain subnetwork N encx The output feature P x_code ;

[0416] Step 3.2: Use subnetwork N ency extract Features;

[0417] Step 3.2.1 Using subnetwork N ency The first layer of Wavelet2 Perform feature extraction to obtain texture information feature P Wavelet2 ;

[0418] Step 3.2.2 Use convolutional layers Conv2_0 and Conv2_1 to sequentially Wavelet2 Perform feature extraction and obtain feature P Conv2_1 ;

[0419] Step 3.2.3 Use the channel_attention2 module to focus on the feature P Conv2_1 Processing includes the following steps a2-c2;

[0420] (a2) Use the parallel GlobalAvgPool2 layer and GlobalMaxPool2 layer to perform feature Conv2_1 Perform pooling operations to obtain features P Globalavg2 and P Globalmax2 ;

[0421] (b2) Use convolutional layers Conv2_3 and Conv2_4 to successively Globalavg2 Perform feature extraction to obtain P Gavgout2 , use convolutional layers Conv2_3 and Conv2_4 to successively Globalmax2 Perform feature extraction to obtain P Gmaxout2 ;

[0422] (c2) For feature P Gavgout2 and PGmaxout2 Perform a summation operation, and then use the nonlinear activation function Sigmoid to calculate the summation result to obtain the weight coefficient P c_a2 ;

[0423] Step 3.2.4 Set the weight coefficient P c_a2 and feature P Conv2_1 Multiply to get feature P channel2 ;

[0424] Step 3.2.5 Use the spatial_attention2 module to focus on feature P channel2 Performing a treatment, comprising the following steps (d2) to (f2);

[0425] (d2) Use the parallel AvgPool2 layer and MaxPool2 layer to process the feature P channel2 Pooling operation is performed along the third dimension to obtain features P avgpool2 and P maxpool2 ;

[0426] (e2) Use the custom connection layer Concat2 to connect P avgpool2 and P maxpool2 Get feature P Concat2 ;

[0427] (f2) Use the convolution layer Conv2_5 to transform the feature P Concat2 Calculate and get the weight coefficient P s_a2 ;

[0428] Step 3.2.6: Set the weight coefficient P s_a2 and feature P channel2 Multiply to get feature P spatial2 ;

[0429] Step 3.2.7 Use the convolutional layer Conv2_2 to transform the feature P spatial2 Perform feature extraction to obtain subnetwork N ency The output feature P y_code ;

[0430] Step 3.3 Use subnetwork N decx For feature P x_code Refactoring

[0431] Step 3.3.1 Using subnetwork N decx The first convolutional layer Conv3_0 has a x_code Perform feature extraction and obtain feature P Conv3_0 ;

[0432] Step 3.3.2 Use the channel_attention3 module to focus on the feature P Conv3_0Performing a treatment, comprising the following steps (a3) ​​to (c3);

[0433] (a3) Use the parallel GlobalAvgPool3 layer and GlobalMaxPool3 layer to perform feature Conv3_0 Perform pooling operations to obtain features P Globalavg3 and P Globalmax3 ;

[0434] (b3) Use convolutional layers Conv3_3 and Conv3_4 to successively process feature P Globalavg3 Perform feature extraction to obtain P Gavgout3 , using convolutional layers Conv3_3 and Conv3_4 to successively Globalmax3 Perform feature extraction to obtain P Gmaxout3 ;

[0435] (c3) For feature P Gavgout3 and P Gmaxout3 Perform a summation operation, and then use the nonlinear activation function Sigmoid to calculate the summation result to obtain the weight coefficient P c_a3 ;

[0436] Step 3.3.3 Set the weight coefficient P c_a3 and feature P Conv3_0 Multiply to get feature P channel3 ;

[0437] Step 3.3.4 Use the spatial_attention3 module to focus on feature P channel3 Performing a treatment, comprising the following steps (d3) to (f3);

[0438] (d3) Use the parallel AvgPool3 layer and MaxPool3 layer to process the feature P channel3 Pooling operation is performed along the third dimension to obtain features P avgpool3 and P maxpool3 ;

[0439] (e3) Use the custom connection layer Concat3 to connect P avgpool3 and P maxpool3 Get feature P Concat3 ;

[0440] (f3) Use the convolution layer Conv3_5 to transform the feature P Concat3 Calculate and get the weight coefficient P s_a3 ;

[0441] Step 3.3.5: Set the weight coefficient P s_a3 and feature P channel3 Multiply to get feature P spatial3 ;

[0442] Step 3.3.6 Use convolutional layers Conv3_1 and Conv3_2 to sequentially spatial3 Perform feature extraction and obtain feature P Conv3_2 ;

[0443] Step 3.3.7 Use the inverse wavelet layer Rewavelet1 to transform the feature P Conv3_2 Reconstruct and get sub-network N decx Output

[0444] Step 3.4: Use subnetwork N decy For feature P y_code Refactoring

[0445] Step 3.4.1 Using subnetwork N decy The first convolutional layer Conv4_0 has a y_code Perform feature extraction and obtain feature P Conv4_0 ;

[0446] Step 3.4.2 Use the channel_attention4 module to focus on the feature P Conv4_0 Performing a treatment, comprising the following steps (a4) to (c4);

[0447] (a4) Use the parallel GlobalAvgPool4 layer and GlobalMaxPool4 layer to perform feature Conv4_0 Perform pooling operations to obtain features P Globalavg4 and P Globalmax4 ;

[0448] (b4) Use convolutional layers Conv4_3 and Conv4_4 to successively Globalavg4 Perform feature extraction to obtain P Gavgout4 , using convolutional layers Conv4_3 and Conv4_4 to successively Globalmax4 Perform feature extraction to obtain P Gmaxout4 ;

[0449] (c4) for feature F Gavgout4 and P Gmaxout4 Perform a summation operation, and then use the nonlinear activation function Sigmoid to calculate the summation result to obtain the weight coefficient P c_a4 ;

[0450] Step 3.4.3 Set the weight coefficient P c_a4 and feature P Conv4_0 Multiply to get feature P channel4 ;

[0451] Step 3.4.4 Use the spatial_attention4 module to focus on feature P channel4 Performing a treatment, comprising the following steps (d4) to (f4);

[0452] (d4) Use the parallel AvgPool4 layer and MaxPool4 layer to process the feature P channel4 Pooling operation is performed along the third dimension to obtain features P avgpool4 and P maxpool4 ;

[0453] (e4) Use the custom connection layer Concat4 to connect P avgpool4 and P maxpool4 Get feature P Concat4 ;

[0454] (f4) Use the convolution layer Conv4_5 to transform the feature P Concat4 Calculate and get the weight coefficient P s_a4 ;

[0455] Step 3.4.5: Set the weight coefficient P s_a4 and feature P channel4 Multiply to get feature P spatial4 ;

[0456] Step 3.4.6 Use convolutional layers Conv4_1 and Conv4_2 to sequentially spatial4 Perform feature extraction and obtain feature P Conv4_2 ;

[0457] Step 3.4.7 Use the inverse wavelet layer Rewavelet2 to transform the feature P Conv4_2 Reconstruct and get sub-network N decy Output

[0458] Step 3.5: Use subnetwork N decx For feature P y_code Reconstruction to obtain X′ P ;

[0459] Step 3.6 Use subnetwork N decy For feature P x_code Reconstruct and obtain Y′ P ;

[0460] Step 3.7 Based on the definitions of formulas (20)-(23), use X′ P and Y′ P Calculate the change detection result map Change_map1;

[0461]

[0462]

[0463]

[0464] Change_map1=Otsu(diff1) (23)

[0465] Among them C X1 is an image The number of channels, C Y1 is an image The number of channels, Otsu(·) represents the Otsu method.

[0466] To verify the effectiveness of the present invention, experiments were carried out using the publicly available Texas dataset, California dataset and Sardinia dataset as examples. The overall accuracy (OA) and Kappa coefficient were used as objective indicators to evaluate the change detection results. The evaluation results of the present invention were compared with those of the ACE-Net method, CAN method, SCCN method, NPSG method and INLPG method.

[0467] The main challenges of change detection tasks are missed classification and misclassification. For change detection based on remote sensing images, detection errors are to classify the changed area as the unchanged area and to classify the unchanged area as the changed area. Figure 4 , Figure 5 and Figure 6It can be seen that compared with other methods, the present invention has a clearer boundary, and compared with the GroundTruth result graph, the present invention has significantly fewer missed points and wrong points, showing a certain degree of superiority. As can be seen from Table 1, for the Texas data set, the OA results obtained by the present invention are 2.8%, 4.7%, 4.8%, 8.1%, and 4.4% higher than those of the ACE-Net method, CAN method, SCCN method, NPSG method, and INLPG method, respectively, with an average increase of 5.0%; the Kappa coefficient results of the present invention are 17.2%, 29.7%, 17.9%, 26.1%, and 29.1% higher than those of the ACE-Net method, CAN method, SCCN method, NPSG method, and INLPG method, respectively, with an average increase of 24.0%. For the California dataset, the OA results obtained by the present invention are improved by 1.8%, 1.2%, 3.4%, 1.3% and 0.6% respectively than those of the ACE-Net method, CAN method, SCCN method, NPSG method and INLPG method, with an average improvement of 1.7%; the Kappa coefficient results are improved by 8.9%, 16.4%, 7.8%, 9.7% and 2.6% respectively than those of the ACE-Net method, CAN method, SCCN method, NPSG method and INLPG method, with an average improvement of 9.1%. For the Sardinia dataset, the OA results obtained by the present invention are improved by 5.3%, 2.6%, 6.3%, 0.8% and 3.7% respectively than those of the ACE-Net method, CAN method, SCCN method, NPSG method and INLPG method, with an average improvement of 3.7%; the Kappa coefficient results of the present invention are improved by 29.2%, 25.3%, 24.2%, 9.5% and 14.8% respectively than those of the ACE-Net method, CAN method, SCCN method, NPSG method and INLPG method, with an average improvement of 20.6%.

[0468] Table 1 Comparison of change detection accuracy (%)

[0469]

[0470] Figure 4 The figures show the detection results of different methods on the Texas dataset, where (a) is the detection result of the ACE-Net method, with an overall accuracy of 94.8%; (b) is the detection result of the CAN method, with an overall accuracy of 92.9%; (c) is the detection result of the SCCN method, with an overall accuracy of 92.8%; (d) is the detection result of the NPSG method, with an overall accuracy of 89.5%; (e) is the detection result of the INLPG method, with an overall accuracy of 93.2%; (f) is the detection result of the present invention, with an overall accuracy of 97.6%; and (g) is the Ground-truth reference figure.

[0471] Figure 5 The figures show the detection results of different methods on the California dataset, where (a) is the detection result of the ACE-Net method, with an overall accuracy of 91.9%; (b) is the detection result of the CAN method, with an overall accuracy of 92.5%; (c) is the detection result of the SCCN method, with an overall accuracy of 90.3%; (d) is the detection result of the NPSG method, with an overall accuracy of 92.4%; (e) is the detection result of the INLPG method, with an overall accuracy of 93.1%; (f) is the detection result of the present invention, with an overall accuracy of 93.7%; and (g) is the Ground-truth reference figure.

[0472] Figure 6 The figures show the detection results of Sardinia dataset using different methods, where (a) is the detection result of ACE-Net method, with an overall accuracy of 90.2%; (b) is the detection result of CAN method, with an overall accuracy of 92.9%; (c) is the detection result of SCCN method, with an overall accuracy of 89.2%; (d) is the detection result of NPSG method, with an overall accuracy of 94.7%; (e) is the detection result of INLPG method, with an overall accuracy of 91.8%; (f) is the detection result of the present invention, with an overall accuracy of 95.5%; and (g) is the Ground-truth reference figure.

[0473] Comprehensive Table 1, Figure 4 , Figure 5 , Figure 6 From the comparison results, it can be seen that the present invention effectively improves the change detection accuracy of heterogeneous remote sensing images by making full use of the texture structure information of heterogeneous remote sensing images and combining the spectral attention mechanism and the spatial attention mechanism.

Claims

1. A heterogeneous remote sensing image change detection method based on topological structure coupling, characterized in that Proceed as follows: Step 1. Establish and initialize the convolutional neural network N for heterogeneous remote sensing image change detection hic , the N hic Contains 2 encoder sub-networks N encx and N ency And 2 decoder sub-networks N decx and N decy ; Step 2: Input the training set H of heterogeneous remote sensing images X and Y to the convolutional neural network N hic Perform unsupervised training; Step 3. Input standardized heterogeneous remote sensing images and And use the convolutional neural network N that has completed unsupervised training hic Complete change detection; The step 1 is specifically as follows: Step 1.1 Establish and initialize the encoder subnetwork N encx , the encoder subnetwork N encx Contains 1 wavelet layer Wavelet1, 3 convolutional layers Conv1_0, Conv1_1 and Conv1_2, 1 channel_attention1 module and 1 spatial_attention1 module; Step 1.

2. Create and initialize the encoder subnetwork N ency , the encoder subnetwork N ency Contains 1 wavelet layer Wavelet2, 3 convolutional layers Conv2_0, Conv2_1 and Conv2_2, 1 channel_attention2 module and 1 spatial_attention2 module; Step 1.

3. Build and initialize the decoder subnetwork N decx , the decoder subnetwork N decx Contains 3 groups of convolutional layers, namely Conv3_0, Conv3_1 and Conv3_2, 1 group of channel_attention3 modules, 1 group of spatial_attention3 modules and 1 group of inverse wavelet layer Rewavelet1; Step 1.

4. Build and initialize the decoder subnetwork N decy , the decoder subnetwork N decy Contains 3 groups of convolutional layers, namely Conv4_0, Conv4_1 and Conv4_2, 1 group of channel_attention4 modules, 1 group of spatial_attention4 modules and 1 group of inverse wavelet layer Rewavelet2; The step 3 comprises: Step 3.1 Use the trained encoder subnetwork N encx extract Features; Step 3.1.1 Using the encoder subnetwork N encx The first layer of Wavelet1 Perform feature extraction to obtain texture information feature P Wavelet1 ; Step 3.1.2 Use convolutional layers Conv1_0 and Conv1_1 to sequentially Wavelet1 Perform feature extraction and obtain feature P Conv1_1 ; Step 3.1.3 Use the channel_attention1 module to focus on the feature P Conv1_1 Processing includes the following steps a1-c1; (a1) Use the parallel GlobalAvgPool1 layer and GlobalMaxPool1 layer to perform feature Conv1_1 Perform pooling operations to obtain features P Globalavg1 and P Globalmax1 ; (b1) Use convolutional layers Conv1_3 and Conv1_4 to sequentially process feature P Globalavg1 Perform feature extraction to obtain P Gavgout1 , using convolutional layers Conv1_3 and Conv1_4 to successively Globalmax1 Perform feature extraction to obtain P Gmaxout1 ; (c1) For feature P Gavgout1 and P Gmaxout1 Perform a summation operation, and then use the nonlinear activation function Sigmoid to calculate the summation result to obtain the weight coefficient P c_a1 ; Step 3.1.4 Set the weight coefficient P c_a1 and feature P Conv1_1 Multiply to get feature P channel1 ; Step 3.1.5 Use the spatial_attention1 module to focus on feature P channel1 Performing a treatment, comprising the following steps (d1) to (f1); (d1) Use the parallel AvgPool1 layer and MaxPool1 layer to process the feature P channel1 Pooling operation is performed along the third dimension to obtain features P avgpool1 and P maxpool1 ; (e1) Use the custom connection layer Concat1 to connect P avgpool1 and P maxpool1 Get feature P Concat1 ; (f1) Use the convolution layer Conv1_5 to transform the feature P Concat1 Calculate and get the weight coefficient P s_a1 ; Step 3.1.6 Set the weight coefficient P s_a1 and feature P channel1 Multiply to get feature P spatial1 ; Step 3.1.7 Use convolutional layer Conv1_2 to transform feature P spatial1 Perform feature extraction to obtain the encoder sub-network N encx The output feature P x_code .

2. The heterogeneous remote sensing image change detection method based on topological structure coupling according to claim 1 is characterized in that The step 2 is specifically as follows: Step 2.

1. Standardize the input remote sensing image X according to formula (1), (2), (3) to obtain Where X = {X i |i=1,…,C1},X i represents the i-th channel of X, represent The i-th channel of , C1 represents the number of channels of X; A=mean(X i )+3×std(X i ) (1) Among them, mean(·) represents the mean of the image, std(·) represents the standard deviation of the image, and max(·) represents the maximum value of the image; Step 2.

2. According to formula (4), (5), (6), the input remote sensing image Y is standardized to obtain Where Y = {Y j |j=1,…,C2},Y j represents the j-th channel of Y, represent The jth channel of , C2 represents the number of channels of Y; B=mean(Y j )+3×std(Y j ) (4) Among them, mean(·) represents the mean of the image, std(·) represents the standard deviation of the image, and max(·) represents the maximum value of the image; Step 2.

3. and Extract pixel point sets from and in express The a-th pixel point in express The ath pixel point in , m represents the total number of selected pixel points; Step 2.

4. Each pixel point is centered on Divide into a series of pixel blocks X of size 100×100 H1 ,by Each pixel point is centered on Divide into a series of pixel blocks Y of size 100×100 H1 ; Step 2.

5. X H1 and Y H1 Each pixel block in is randomly rotated z×90 degrees counterclockwise with its center point as the rotation center to obtain the pixel block set X H2 and Y H2 , where z represents a value randomly selected from the set {1,2,3,4}; Step 2.

6. X H1 and Y H1 Each pixel block in is flipped upside down to obtain the pixel block set X H3 and Y H3 ; Step 2.

7. Order Will and As the training set of the change detection neural network, it is fed into the convolutional neural network N hic Perform unsupervised training, where and represents the pixel block in the training set, set the number of iterations iter←1, and execute steps 2.8 to 2.19; Step 2.8 uses the encoder subnetwork N encx extract Features; Step 2.8.1 Using the encoder subnetwork N encx The first layer of Wavelet1 Perform feature extraction to obtain texture information feature F Wavelet1 ; Step 2.8.2 Use convolutional layers Conv1_0 and Conv1_1 to perform F Wavelet1 Perform feature extraction and obtain feature F Conv1_1 ; Step 2.8.3 Use the channel_attention1 module to focus on the feature F Conv1_1 Performing a treatment, comprising the following steps (a1) to (c1); (a1) Use the parallel GlobalAvgPool1 layer and GlobalMaxPool1 layer to perform feature Conv1_1 Perform pooling operations to obtain features F Globalavg1 and F Globalmax1 ; (b1) Use convolutional layers Conv1_3 and Conv1_4 to sequentially process feature F Globalavg1 Perform feature extraction to obtain F Gavgout1 , using convolutional layers Conv1_3 and Conv1_4 to successively Globalmax1 Perform feature extraction to obtain F Gmaxout1 ; (c1) for feature F Gavgout1 and F Gmaxout1 Perform a summation operation, and then use the nonlinear activation function Sigmoid to calculate the summation result to obtain the weight coefficient F c_a1 ; Step 2.8.4: Set the weight coefficient F c_a1 and feature F Conv1_1 Multiply to get feature F channel1 ; Step 2.8.5 Use the spatial_attention1 module to focus on feature F channel1 Performing a treatment, comprising the following steps (d1) to (f1); (d1) Use the parallel AvgPool1 layer and MaxPool1 layer to process the feature F channel1 Pooling operation is performed along the third dimension to obtain features F avgpool1 and F maxpool1 ; (e1) Use the custom connection layer Concat1 to connect F avgpool1 and F maxpool1 Get feature F Concat1 ; (f1) Use the convolution layer Conv1_5 to transform the feature F Concat1 Calculate and get the weight coefficient F s_a1 ; Step 2.8.6: Set the weight coefficient F s_a1 and feature F channel1 Multiply to get feature F spatial1 ; Step 2.8.7 Use convolutional layer Conv1_2 to transform feature F spatial1 Perform feature extraction to obtain the encoder sub-network N encx The output feature F x_code ; Step 2.9 uses the encoder subnetwork N ency extract Features; Step 2.9.1 Using the encoder subnetwork N ency The first layer of Wavelet2 Perform feature extraction to obtain texture information feature F Wavelet2 ; Step 2.9.2 Use convolutional layers Conv2_0 and Conv2_1 to perform F Wavelet2 Perform feature extraction and obtain feature F Conv2_1 ; Step 2.9.3 Use the channel_attention2 module to focus on the feature F Conv2_1 Performing a treatment, comprising the following steps (a2) to (c2); (a2) Use the parallel GlobalAvgPool2 layer and GlobalMaxPool2 layer to perform feature Conv2_1 Perform pooling operations to obtain features F Globalavg2 and F Globalmax2 ; (b2) Use convolutional layers Conv2_3 and Conv2_4 to successively process feature F Globalavg2 Perform feature extraction to obtain F Gavgout2 , using convolutional layers Conv2_3 and Conv2_4 to successively Globalmax2 Perform feature extraction to obtain F Gmaxout2 ; (c2) for feature F Gavgout2 and F Gmaxout2 Perform a summation operation, and then use the nonlinear activation function Sigmoid to calculate the summation result to obtain the weight coefficient F c_a2 ; Step 2.9.4: Set the weight coefficient F c_a2 and feature F Conv2_1 Multiply to get feature F channel2 ; Step 2.9.5 Use the spatial_attention2 module to focus on feature F channel2 Performing a treatment, comprising the following steps (d2) to (f2); (d2) Use the parallel AvgPool2 layer and MaxPool2 layer to process the feature F channel2 Pooling operation is performed along the third dimension to obtain features F avgpool2 and F maxpool2 ; (e2) Use the custom connection layer Concat2 to connect F avgpool2 and F maxpool2 Get feature F Concat2 ; (f2) Use the convolution layer Conv2_5 to perform feature F Concat2 Calculate and get the weight coefficient F s_a2 ; Step 2.9.6: Set the weight coefficient F s_a2 and feature F channel2 Multiply to get feature F spatial2 ; Step 2.9.7 Use the convolutional layer Conv2_2 to transform the feature F spatial2 Perform feature extraction to obtain the encoder sub-network N ency The output feature F y_code ; Step 2.10 uses the decoder subnetwork N decx For feature F x_code Refactoring Step 2.10.1 Using the decoder subnetwork N decx The first convolutional layer Conv3_0 is used for F x_code Perform feature extraction and obtain feature F Conv3_0 ; Step 2.10.2 Use the channel_attention3 module to focus on the feature F Conv3_0 Performing a treatment, comprising the following steps (a3) ​​to (c3); (a3) Use the parallel GlobalAvgPool3 layer and GlobalMaxPool3 layer to perform feature Conv3_0 Perform pooling operations to obtain features F Globalavg3 and F Globalmax3 ; (b3) Use convolutional layers Conv3_3 and Conv3_4 to successively Globalavg3 Perform feature extraction to obtain F Gavgout3 , using convolutional layers Conv3_3 and Conv3_4 to successively Globalmax3 Perform feature extraction to obtain F Gmaxout3 ; (c3) for feature F Gavgout3 and F Gmaxout3 Perform a summation operation, and then use the nonlinear activation function Sigmoid to calculate the summation result to obtain the weight coefficient F c_a3 ; Step 2.10.3: Set the weight coefficient F c_a3 and feature F Conv3_0 Multiply to get feature F channel3 ; Step 2.10.4 Use the spatial_attention3 module to focus on feature F channel3 Performing a treatment, comprising the following steps (d3) to (f3); (d3) Use parallel AvgPool3 and MaxPool3 layers to process feature F channel3 Pooling operation is performed along the third dimension to obtain features F avgpool3 and F maxpool3 ; (e3) Use the custom connection layer Concat3 to connect F avgpool3 and F maxpool3 Get feature F Concat3 ; (f3) Use the convolution layer Conv3_5 to transform the feature F Concat3 Calculate and get the weight coefficient F s_a3 ; Step 2.10.5: Set the weight coefficient F s_a3 and feature F channel3 Multiply to get feature F spatial3 ; Step 2.10.6 Use convolutional layers Conv3_1 and Conv3_2 to perform F spatial3 Perform feature extraction and obtain feature F Conv3_2 ; Step 2.10.7 Use the inverse wavelet layer Rewavelet1 to transform the feature F Conv3_2 Reconstruct and get the decoder subnetwork N decx Output Step 2.11 uses the decoder subnetwork N decy For feature F y_code Refactoring Step 2.11.1 Using the decoder subnetwork N decy The first convolutional layer Conv4_0 is used for F y_code Perform feature extraction and obtain feature F Conv4_0 ; Step 2.11.2 Use the channel_attention4 module to focus on the feature F Conv4_0 Performing a treatment, comprising the following steps (a4) to (c4); (a4) Use the parallel GlobalAvgPool4 layer and GlobalMaxPool4 layer to perform feature Conv4_0 Perform pooling operations to obtain features F Globalavg4 and F Globalmax4 ; (b4) Use convolutional layers Conv4_3 and Conv4_4 to successively Globalavg4 Perform feature extraction to obtain F Gavgout4 , using convolutional layers Conv4_3 and Conv4_4 to successively Globalmax4 Perform feature extraction to obtain F Gmaxout4 ; (c4) for feature F Gavgout4 and F Gmaxout4 Perform a summation operation, and then use the nonlinear activation function Sigmoid to calculate the summation result to obtain the weight coefficient F c_a4 ; Step 2.11.3: Set the weight coefficient F c_a4 and feature F Conv4_0 Multiply to get feature F channel4 ; Step 2.11.4 Use the spatial_attention4 module to focus on feature F channel4 Performing a treatment, comprising the following steps (d4) to (f4); (d4) Use parallel AvgPool4 and MaxPool4 layers to process feature F channel4 Pooling operation is performed along the third dimension to obtain features F avgpool4 and F maxpool4 ; (e4) Use the custom connection layer Concat4 to connect F avgpool4 and F maxpool4 Get feature F Concat4 ; (f4) Use the convolution layer Conv4_5 to perform feature F Concat4 Calculate and get the weight coefficient F s_a4 ; Step 2.11.5: Set the weight coefficient F s_a4 and feature F channel4 Multiply to get feature F spatial4 ; Step 2.11.6 Use convolutional layers Conv4_1 and Conv4_2 to perform F spatial4 Perform feature extraction and obtain feature F Conv4_2 ; Step 2.11.7 Use the inverse wavelet layer Rewavelet2 to transform the feature F Conv4_2 Reconstruct and get the decoder subnetwork N decy Output Step 2.12 uses the decoder subnetwork N decx For feature F y_code Reconstruction to obtain X' H ; Step 2.13 uses the decoder subnetwork N decy For feature F x_code Reconstruct to get Y' H ; Step 2.14 uses the encoder subnetwork N encx X' H Perform feature extraction to obtain F' x_code , and then use the decoder sub-network N decy For feature F' x_code Reconstruction to obtain Step 2.15: Use encoder subnetwork N ency Y' H Perform feature extraction to obtain F' y_code , and then use the decoder sub-network N decx For feature F' y_code Reconstruction to obtain Step 2.16 Based on the definitions of formulas (7)-(10), use X' H and Y' H Calculate the change detection result map Change_map; Among them C X is an image The number of channels, C Y is an image The number of channels, Otsu(·) represents the Otsu method; Step 2.17 According to the definition of formulas (11)-(15), use the loss function L enc For the encoder sub-network N encx and N ency Update parameters; Where affinity(·) represents the affinity matrix of the image, C x_code Indicates F x_code The number of channels, M represents the number of pixels of S; Step 2.18 According to the definitions of formulas (16)-(19), use the loss function L to train the convolutional neural network N hic Update parameters; L=α1L1+α2L2+α3L3 (19) Where N represents The number of pixels, clw represents the change prior, α1, α2 and α3 represent the preset weight coefficients; Step 2.19: If all pixel blocks in the training set have been processed, go to step 2.20; otherwise, take a group of unprocessed pixel blocks from the training set and return to step 2.8; Step 2.20 Let iter←iter+1. If the number of iterations iter>Total_iter, the trained neural network N is obtained. hic , go to step 3; otherwise, use the Adam-based back-error propagation algorithm and the prediction loss L enc and L updates N hic , go to step 2.8 to reprocess all pixel blocks in the training set, and Total_iter represents the preset number of iterations.

3. The heterogeneous remote sensing image change detection method based on topological structure coupling according to claim 2 is characterized in that The step 3 also includes: Step 3.2: Use the trained encoder subnetwork N ency extract Features; Step 3.2.1 Using the encoder subnetwork N ency The first layer of Wavelet2 Perform feature extraction to obtain texture information feature P Wavelet2 ; Step 3.2.2 Use convolutional layers Conv2_0 and Conv2_1 to sequentially Wavelet2 Perform feature extraction and obtain feature P Conv2_1 ; Step 3.2.3 Use the channel_attention2 module to focus on the feature P Conv2_1 Processing includes the following steps a2-c2; (a2) Use the parallel GlobalAvgPool2 layer and GlobalMaxPool2 layer to perform feature Conv2_1 Perform pooling operations to obtain features P Globalavg2 and P Globalmax2 ; (b2) Use convolutional layers Conv2_3 and Conv2_4 to successively Globalavg2 Perform feature extraction to obtain P Gavgout2 , use convolutional layers Conv2_3 and Conv2_4 to successively Globalmax2 Perform feature extraction to obtain P Gmaxout2 ; (c2) For feature P Gavgout2 and P Gmaxout2 Perform a summation operation, and then use the nonlinear activation function Sigmoid to calculate the summation result to obtain the weight coefficient P c_a2 ; Step 3.2.4 Set the weight coefficient P c_a2 and feature P Conv2_1 Multiply to get feature P channel2 ; Step 3.2.5 Use the spatial_attention2 module to focus on feature P channel2 Performing a treatment, comprising the following steps (d2) to (f2); (d2) Use the parallel AvgPool2 layer and MaxPool2 layer to process the feature P channel2 Pooling operation is performed along the third dimension to obtain features P avgpool2 and P maxpool2 ; (e2) Use the custom connection layer Concat2 to connect P avgpool2 and P maxpool2 Get feature P Concat2 ; (f2) Use the convolution layer Conv2_5 to transform the feature P Concat2 Calculate and get the weight coefficient P s_a2 ; Step 3.2.6: Set the weight coefficient P s_a2 and feature P channel2 Multiply to get feature P spatial2 ; Step 3.2.7 Use the convolutional layer Conv2_2 to transform the feature P spatial2 Perform feature extraction to obtain the encoder sub-network N ency The output feature P y_code ; Step 3.3: Use the trained decoder subnetwork N decx For feature P x_code Refactoring Step 3.3.1 Using the decoder subnetwork N decx The first convolutional layer Conv3_0 has a x_code Perform feature extraction and obtain feature P Conv3_0 ; Step 3.3.2 Use the channel_attention3 module to focus on the feature P Conv3_0 Performing a treatment, comprising the following steps (a3) ​​to (c3); (a3) Use the parallel GlobalAvgPool3 layer and GlobalMaxPool3 layer to perform feature Conv3_0 Perform pooling operations to obtain features P Globalavg3 and P Globalmax3 ; (b3) Use convolutional layers Conv3_3 and Conv3_4 to successively process feature P Globalavg3 Perform feature extraction to obtain P Gavgout3 , using convolutional layers Conv3_3 and Conv3_4 to successively Globalmax3 Perform feature extraction to obtain P Gmaxout3 ; (c3) For feature P Gavgout3 and P Gmaxout3 Perform a summation operation, and then use the nonlinear activation function Sigmoid to calculate the summation result to obtain the weight coefficient P c_a3 ; Step 3.3.3 Set the weight coefficient P c_a3 and feature P Conv3_0 Multiply to get feature P channel3 ; Step 3.3.4 Use the spatial_attention3 module to focus on feature P channel3 Performing a treatment, comprising the following steps (d3) to (f3); (d3) Use the parallel AvgPool3 layer and MaxPool3 layer to process the feature P channel3 Pooling operation is performed along the third dimension to obtain features P avgpool3 and P maxpool3 ; (e3) Use the custom connection layer Concat3 to connect P avgpool3 and P maxpool3 Get feature P Concat3 ; (f3) Use the convolution layer Conv3_5 to transform the feature P Concat3 Calculate and get the weight coefficient P s_a3 ; Step 3.3.5: Set the weight coefficient P s_a3 and feature P channel3 Multiply to get feature P spatial3 ; Step 3.3.6 Use convolutional layers Conv3_1 and Conv3_2 to sequentially spatial3 Perform feature extraction and obtain feature P Conv3_2 ; Step 3.3.7 Use the inverse wavelet layer Rewavelet1 to transform the feature P Conv3_2 Reconstruct and get the decoder subnetwork N decx Output Step 3.4 uses the trained decoder subnetwork N decy For feature P y_code Refactoring Step 3.4.1 Using the decoder subnetwork N decy The first convolutional layer Conv4_0 has a y_code Perform feature extraction and obtain feature P Conv4_0 ; Step 3.4.2 Use the channel_attention4 module to focus on the feature P Conv4_0 Performing a treatment, comprising the following steps (a4) to (c4); (a4) Use the parallel GlobalAvgPool4 layer and GlobalMaxPool4 layer to perform feature Conv4_0 Perform pooling operations to obtain features P Globalavg4 and P Globalmax4 ; (b4) Use convolutional layers Conv4_3 and Conv4_4 to successively Globalavg4 Perform feature extraction to obtain P Gavgout4 , using convolutional layers Conv4_3 and Conv4_4 to successively Globalmax4 Perform feature extraction to obtain P Gmaxout4 ; (c4) For feature P Gavgout4 and P Gmaxout4 Perform a summation operation, and then use the nonlinear activation function Sigmoid to calculate the summation result to obtain the weight coefficient P c_a4 ; Step 3.4.3 Set the weight coefficient P c_a4 and feature P Conv4_0 Multiply to get feature P channel4 ; Step 3.4.4 Use the spatial_attention4 module to focus on feature P channel4 Performing a treatment, comprising the following steps (d4) to (f4); (d4) Use the parallel AvgPool4 layer and MaxPool4 layer to process the feature P channel4 Pooling operation is performed along the third dimension to obtain features P avgpool4 and P maxpool4 ; (e4) Use the custom connection layer Concat4 to connect P avgpool4 and P maxpool4 Get feature P Concat4 ; (f4) Use the convolution layer Conv4_5 to transform the feature P Concat4 Calculate and get the weight coefficient P s_a4 ; Step 3.4.5: Set the weight coefficient P s_a4 and feature P channel4 Multiply to get feature P spatial4 ; Step 3.4.6 Use convolutional layers Conv4_1 and Conv4_2 to sequentially spatial4 Perform feature extraction and obtain feature P Conv4_2 ; Step 3.4.7 Use the inverse wavelet layer Rewavelet2 to transform the feature P Conv4_2 Reconstruct and get the decoder subnetwork N decy Output Step 3.5: Use the decoder subnetwork N decx For feature P y_code Reconstruction to obtain X' P ; Step 3.6 uses the decoder subnetwork N decy For feature P x_code Reconstruct to get Y' P ; Step 3.7 Based on the definitions of formulas (20)-(23), use X' P and Y' P Calculate the change detection result map Change_map1; Change_map1=Otsu(diff1) (23) Among them C X1 is an image The number of channels, C Y1 is an image The number of channels, Otsu(·) represents the Otsu method.

Images