A remote sensing image change detection method based on a modular learning network

Abstract

The application provides a remote sensing image change detection method based on a modular learning network in the field of change detection, which decouples the change detection method, and regards the change detection as the change of the object related to the detection task. Specifically, in the framework, two special components are designed: a task-centered learner and a difference detector. The task-centered learner is based on an encoder-decoder structure and is committed to identifying the object related to the task. The difference detector combines a difference module and a feature pyramid network and is used for identifying the change between targets. The method can greatly improve the prediction accuracy of the network.

Description

Technical Field

[0001] This invention belongs to the field of deep learning in artificial intelligence, specifically a method for detecting changes in remote sensing images based on modular learning networks. Background Technology

[0002] Change detection is an interdisciplinary field between computer vision and remote sensing, and it is also fundamental to some advanced computer vision tasks. It has significant implications for ecological protection, environmental monitoring, and land resource utilization. Change detection involves effectively predicting regions where changes have occurred between two RGB images.

[0003] Despite the rapid development in the field of change detection, the current mainstream single-stream and dual-stream methods both require the simultaneous implementation of task-related object detection and change region detection in the decoder, resulting in low accuracy in practical applications.

[0004] Therefore, how to provide a change detection method that can improve the accuracy of networks without requiring complex structural design, and improve the performance of existing models, has become an urgent problem to be solved. Summary of the Invention

[0005] The technical problem to be solved by this invention is to provide a remote sensing image change detection method based on a modular learning network, while improving the accuracy and efficiency of existing change detection methods.

[0006] This invention provides a method for detecting changes in remote sensing images based on modular learning networks, the method comprising the following steps:

[0007] Step S10: Build a modular learning network based on Res2Net.

[0008] Step S20: Obtain labeled sample images to construct a dataset, and use the sample images in the dataset to train the modular learning network.

[0009] Step S30: Input the image to be tested into the trained modular learning network. The modular learning network outputs the probability value corresponding to the region. If the probability value is greater than or equal to the threshold, the region is marked as a changed region; otherwise, it is marked as an unchanged region.

[0010] Furthermore, the modular learning network in step S10 includes:

[0011] A 3×3 first convolution kernel, a 3×3 second convolution kernel, a 3×3 third convolution kernel, a 3×3 fourth convolution kernel, a 3×3 fifth convolution kernel, a 3×3 sixth convolution kernel, a 3×3 seventh convolution kernel, a 3×3 eighth convolution kernel, a 1×1 ninth convolution kernel, a 1×1 tenth convolution kernel, a 1×1 eleventh convolution kernel, a 1×1 twelfth convolution kernel, and a channel. The network consists of a first ReLU activation function with 64 channels, a second ReLU activation function with 64 channels, a third ReLU activation function with 64 channels, a first subnet with 256 channels, a second subnet with 512 channels, a third subnet with 1024 channels, a fourth subnet with 2048 channels, a first difference module with 256 channels, a second difference module with 128 channels, a third difference module with 64 channels, and a fourth difference module with 32 channels.

[0012] The first subnet includes three horizontally parallel and connected multi-scale bottleneck modules, sequentially labeled with S... 1,1 S 1,2 S 1,3 The second subnet comprises four horizontally parallel and connected multi-scale bottleneck modules, sequentially labeled S. 2,1 S 2,2 ... S 2,4 The third subnet comprises six horizontally parallel and interconnected multi-scale bottleneck modules, sequentially labeled S. 3,1 S 3,2 ... S 3,6 The fourth subnet comprises three horizontally parallel and connected multi-scale bottleneck modules, sequentially labeled S. 4,1 S 4,2 S 4,3 express.

[0013] The output of the first convolutional kernel is connected to the input of the first ReLU activation function, the output of the first ReLU activation function is connected to the input of the second convolutional kernel, and the output of the second convolutional kernel is connected to the input of the second ReLU activation function, which is used to change the number of channels in the image feature map; the output of the second ReLU activation function is connected to the input of the first sub-network, which is used to extract image features; the first sub-network, the second sub-network, the third sub-network, and the fourth sub-network are arranged from left to right in sequence.

[0014] The first subnet outputs feature map S out,1 The second subnet outputs feature map S out,2 The third subnet outputs feature map S out,3The fourth subnet outputs feature map S out,4 The feature map S out,4 After performing one bilinear interpolation upsampling operation, the resolution of the feature map is increased to match that of feature map S. out,3 Consistent with and consistent with feature map S out,3 Element stacking is performed to form feature map F out,3 ; in the feature map F out,3 The feature map F is then obtained by performing convolution operations with the third and fourth convolution kernels. out,up3 ; the feature map F out,up3 After performing one bilinear interpolation upsampling operation, the resolution of the feature map is increased to match that of feature map S. out,2 Consistent with and consistent with feature map S out,2 The feature map F is obtained by stacking elements. out,2 ; in the feature map F out,2 Then, the feature map F is obtained by performing convolution operations with the fifth and sixth convolution kernels. out,up2 ; the feature map F out,up2 After performing one bilinear interpolation upsampling operation, the resolution of the feature map is increased to match that of feature map S. out,1 Consistent with and consistent with feature map S out,1 The feature map F is obtained by stacking elements. out,1 ; in the feature map F out,1 The feature map F is then obtained by performing convolution operations with the seventh and eighth convolution kernels. out,up1 .

[0015] The feature map S out,4 The feature map F is obtained by sending it into the first difference module. diff,4 The feature map F out,up3 The feature map F is obtained by performing a convolution operation on the output of the second difference module after inputting it into the ninth convolution kernel. f,up3 ; the feature map F diff,4 After performing one bilinear interpolation upsampling operation, the resolution of the feature map is increased to be comparable to that of feature map F. f,up3 Consistent with and consistent with feature map F f,up3 The feature map F is obtained by adding elements together. p,3 ; in the feature map F out,up2 The feature map F is obtained by performing a convolution operation on the output of the third difference module after inputting it into the tenth convolution kernel. f,up2 ; in the feature map F p,3 After performing one bilinear interpolation upsampling operation, the resolution of the feature map is increased to be comparable to that of feature map F. f,up2 Consistent with and consistent with feature map F f,up2 The feature map F is obtained by adding elements together. p,2 ; in the feature map F out,up1After inputting the fourth difference module, the output undergoes an eleventh convolution operation to obtain the feature map F. f,up1 ; in the feature map F p,2 After performing one bilinear interpolation upsampling, the resolution of the feature map is increased to be comparable to that of feature map F. f,up1 Consistent with and consistent with feature map F f,up1 The feature map F is obtained by adding elements together. p,1 ; in the feature map F p,1 Then, the twelfth convolution kernel operation is performed to obtain the feature map F. out .

[0016] Furthermore, in step S20, the process of training the modular learning network includes:

[0017] Step S21: Obtain sample images from the dataset, adjust the sample images to RGB images of size 256×256, and then input them into the constructed modular learning network.

[0018] Step S22: The modular learning network outputs feature maps F. out .

[0019] Step S23: Calculate the L of the modular learning network using the binary classification cross-quotient loss function and the real labels. bce Loss value:

[0020]

[0021] Where N represents all pixels in the sample image; y i Represents the actual label value, y i ∈{0,1}, This represents the probability value of the region of change predicted by the modular learning network.

[0022] Calculate the Dice loss value of the modular learning network using the Dice loss function and the true labels:

[0023]

[0024] Where N represents all pixels in the sample image; y i Represents the actual label value, y i ∈{0,1}, y represents the probability value predicted by the network. i ∈[0,1], where ∈ represents a constant.

[0025] Weighted L bce The total loss L of the modular learning network is obtained from the loss value and the Dice loss value:

[0026] L=αLbce +βL dice

[0027] Where α and β are weights, α = β = 1.

[0028] Step S24: Update the parameters of the modular learning network in reverse using gradient descent based on the total loss L until the total loss L reaches the preset threshold, at which point the training ends.

[0029] Furthermore, step S30 includes:

[0030] After adjusting the image to be tested to a 256×256 RGB image, it is input into the trained modular learning network to calculate the probability value of the regions in the RGB image; regions with a probability value greater than or equal to 0.5 are marked as changed regions, and regions with a probability value less than 0.5 are marked as unchanged regions.

[0031] Compared with the prior art, the beneficial effects of the present invention are:

[0032] By decoupling the change detection task into detecting task-related features and identifying changes, each module in the network is ensured to focus on only one subtask. The network designed based on this principle simplifies the entire change detection process and improves model performance and accuracy. Furthermore, this invention is model-independent, allowing existing methods to be ported to the framework proposed in this invention, thereby improving model accuracy. Attached Figure Description

[0033] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0034] Figure 1 This is a flowchart of the method of the present invention.

[0035] Figure 2 This is a schematic diagram of the structure of a modular learning network.

[0036] Figure 3 This is a structural diagram of a multi-scale bottleneck module.

[0037] Figure 4 This is a structural diagram of the difference module. Detailed Implementation

[0038] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0039] To better understand the technical solution of the present invention, the above technical solution will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0040] Example 1: A method for detecting changes in remote sensing images based on modular learning networks, such as... Figure 1 The method includes the following steps:

[0041] Step S10: Build a modular learning network based on Res2Net.

[0042] Step S20: Obtain labeled sample images to construct a dataset, and use the sample images in the dataset to train the modular learning network.

[0043] This involves acquiring a large number of sample images beforehand, labeling the changing regions of each sample image, and dividing them into training, validation, and test sets. The sample images in the training set are then input into the modular learning network for training. The validation set is then used to validate the trained modular learning network and determine whether the loss value reaches a preset threshold. Finally, the test set is input into the trained modular learning network to determine the capabilities achieved by the modular learning network.

[0044] Step S30: Input the image to be tested into the trained modular learning network and calculate the final changed region.

[0045] For a better option, please refer to the following. Figure 2 The modular learning network includes:

[0046] A 3×3 first convolution kernel, a 3×3 second convolution kernel, a 3×3 third convolution kernel, a 3×3 fourth convolution kernel, a 3×3 fifth convolution kernel, a 3×3 sixth convolution kernel, a 3×3 seventh convolution kernel, a 3×3 eighth convolution kernel, a 1×1 ninth convolution kernel, a 1×1 tenth convolution kernel, a 1×1 eleventh convolution kernel, a 1×1 twelfth convolution kernel, and a channel. The network consists of a first ReLU activation function with 64 channels, a second ReLU activation function with 64 channels, a third ReLU activation function with 64 channels, a first subnet with 256 channels, a second subnet with 512 channels, a third subnet with 1024 channels, a fourth subnet with 2048 channels, a first difference module with 256 channels, a second difference module with 128 channels, a third difference module with 64 channels, and a fourth difference module with 32 channels.

[0047] The first subnet includes three horizontally parallel and connected multi-scale bottleneck modules, sequentially labeled with S... 1,1 S 1,2S 1,3 The second subnet comprises four horizontally parallel and connected multi-scale bottleneck modules, sequentially labeled S. 2,1 S 2,2 ... S 2,4 The third subnet comprises six horizontally parallel and interconnected multi-scale bottleneck modules, sequentially labeled S. 3,1 S 3,2 ... S 3,6 The fourth subnet comprises three horizontally parallel and connected multi-scale bottleneck modules, sequentially labeled S. 4,1 S 4,2 S 4,3 express.

[0048] The output of the first convolutional kernel is connected to the input of the first ReLU activation function, the output of the first ReLU activation function is connected to the input of the second convolutional kernel, and the output of the second convolutional kernel is connected to the input of the second ReLU activation function, which is used to change the number of channels in the image feature map; the output of the second ReLU activation function is connected to the input of the first sub-network, which is used to extract image features; the first sub-network, the second sub-network, the third sub-network, and the fourth sub-network are arranged from left to right in sequence.

[0049] The first subnet outputs feature map S out,1 The second subnet outputs feature map S out,2 The third subnet outputs feature map S out,3 The fourth subnet outputs feature map S out,4 The feature map S out,4 After performing one bilinear interpolation upsampling operation, the resolution of the feature map is increased to match that of feature map S. out,3 Consistent with and consistent with feature map S out,3 Element stacking is performed to form feature map F out,3 ; in the feature map F out,3 The feature map F is then obtained by performing convolution operations with the third and fourth convolution kernels. out,up3 ; the feature map F out,up3 After performing one bilinear interpolation upsampling operation, the resolution of the feature map is increased to match that of feature map S. out,2 Consistent with and consistent with feature map S out,2 The feature map F is obtained by stacking elements. out,2 ; in the feature map F out,2 Then, the feature map F is obtained by performing convolution operations with the fifth and sixth convolution kernels. out,up2 ; the feature map F out,up2 After performing one bilinear interpolation upsampling operation, the resolution of the feature map is increased to match that of feature map S. out,1 Consistent with and consistent with feature map Sout,1 The feature map F is obtained by stacking elements. out,1 ; in the feature map F out,1 The feature map F is then obtained by performing convolution operations with the seventh and eighth convolution kernels. out,up1 .

[0050] The feature map S out,4 The feature map F is obtained by sending it into the first difference module. diff,4 The feature map F out,up3 The feature map F is obtained by performing a convolution operation on the output of the second difference module after inputting it into the ninth convolution kernel. f,up3 ; the feature map F diff,4 After performing one bilinear interpolation upsampling operation, the resolution of the feature map is increased to be comparable to that of feature map F. f,up3 Consistent with and consistent with feature map F f,up3 The feature map F is obtained by adding elements together. p,3 ; in the feature map F out,up2 The feature map F is obtained by performing a convolution operation on the output of the third difference module after inputting it into the tenth convolution kernel. f,up2 ; in the feature map F p,3 After performing one bilinear interpolation upsampling operation, the resolution of the feature map is increased to be comparable to that of feature map F. f,up2 Consistent with and consistent with feature map F f,up2 The feature map F is obtained by adding elements together. p,2 ; in the feature map F out,up1 After inputting the fourth difference module, the output undergoes an eleventh convolution operation to obtain the feature map F. f,up1 ; in the feature map F p,2 After performing one bilinear interpolation upsampling, the resolution of the feature map is increased to be comparable to that of feature map F. f,up1 Consistent with and consistent with feature map F f,up1 The feature map F is obtained by adding elements together. p,1 ; in the feature map F p,1 Then, the twelfth convolution kernel operation is performed to obtain the feature map F. out .

[0051] Preferably, in step S21, the sample images in the dataset are obtained, and after the sample images are adjusted to RGB images of size 256×256, they are input into the constructed modular learning network.

[0052] Step S22: The modular learning network outputs feature maps F. out ;

[0053] Step S23: Calculate the L of the modular learning network using the binary classification cross-quotient loss function and the real labels. bce Loss value:

[0054]

[0055] Where N represents all pixels in the sample image; y i Represents the actual label value, y i ∈{0,1}, This represents the probability value of the region of change predicted by the modular learning network.

[0056] Calculate the Dice loss value of the modular learning network using the Dice loss function and the true labels:

[0057]

[0058] Where N represents all pixels in the sample image; y i Represents the actual label value, y i ∈{0,1}, y represents the probability value predicted by the network. i ∈[0,1], where ∈ represents a constant to prevent the denominator from being 0. In this invention, the value is 0.000001.

[0059] Weighted L bce The total loss L of the modular learning network is obtained from the loss value and the Dice loss value:

[0060] L=αL bce +βL dice

[0061] Where α and β are weights, α = β = 1.

[0062] Step S24: Update the parameters of the modular learning network in reverse using gradient descent based on the total loss L until the total loss L reaches the preset threshold, at which point the training ends.

[0063] Preferably, step S30 includes:

[0064] After adjusting the image to be tested to a 256×256 RGB image, it is input into the trained modular learning network.

[0065] Calculate the probability value of regions in an RGB image; mark regions with a probability value greater than or equal to 0.5 as changed regions, and mark regions with a probability value less than 0.5 as unchanged regions.

[0066] Experimental Analysis

[0067] Table 1 Performance comparison on the SYSU test set

[0068] Comparison Methods F1 IoU OA BIT 78.15 64.13 90.18 HCGMNet 79.76 66.33 91.12 DMINet 80.30 67.09 91.30 This invention 82.39 70.05 91.94

[0069] Table 1 presents the experimental data on the SYSU test set, comparing the performance of classic change detection methods on three evaluation metrics: F1, IoU, and OA. Our invention achieves state-of-the-art results across multiple metrics. Compared to the BIT method, our invention improves F1 and IoU by 5.4% and 9.2%, respectively. Compared to the HCGMNet method, our invention improves F1 and IoU by 2.63 and 2.72 percentage points, respectively. Compared to the state-of-the-art DMINet method, our invention surpasses DMINet on all metrics, particularly improving F1 by 2.09 percentage points. These experimental results demonstrate that our invention outperforms existing change detection algorithms. These substantial improvements can be attributed to the modular learning relationships proposed in our invention. These enhancements validate the effectiveness and superiority of our method.

[0070] The core idea of ​​this invention is a modular learning method. Specifically, change detection is viewed as a combined task of locating task-related objects and detecting changes. A task-related detector is built based on a Res2Net network, and then the proposed difference detector is used to detect specific changes. Extensive experiments on highly challenging datasets have validated the superior performance and advantages of the proposed method.

[0071] While specific embodiments of the present invention have been described above, those skilled in the art should understand that the specific embodiments described are merely illustrative and not intended to limit the scope of the present invention. Equivalent modifications and variations made by those skilled in the art in accordance with the spirit of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for detecting changes in remote sensing images based on modular learning networks, characterized in that: The method includes the following steps: Step S10: Build a modular learning network based on Res2Net, with the following structure: A 3×3 first convolution kernel, a 3×3 second convolution kernel, a 3×3 third convolution kernel, a 3×3 fourth convolution kernel, a 3×3 fifth convolution kernel, a 3×3 sixth convolution kernel, a 3×3 seventh convolution kernel, a 3×3 eighth convolution kernel, a 1×1 ninth convolution kernel, a 1×1 tenth convolution kernel, a 1×1 eleventh convolution kernel, a 1×1 twelfth convolution kernel, and a channel. The network consists of a first ReLU activation function with 64 channels, a second ReLU activation function with 64 channels, a third ReLU activation function with 64 channels, a first subnet with 256 channels, a second subnet with 512 channels, a third subnet with 1024 channels, a fourth subnet with 2048 channels, a first difference module with 256 channels, a second difference module with 128 channels, a third difference module with 64 channels, and a fourth difference module with 32 channels. The first subnet includes three horizontally parallel and connected multi-scale bottleneck modules, sequentially labeled with S... 1,1 S 1,2 S 1,3 The second subnet comprises four horizontally parallel and connected multi-scale bottleneck modules, sequentially labeled S. 2,1 S 2,2 ... S 2,4 The third subnet comprises six horizontally parallel and interconnected multi-scale bottleneck modules, sequentially labeled S. 3,1 S 3,2 ... S 3,6 The fourth subnet comprises three horizontally parallel and connected multi-scale bottleneck modules, sequentially labeled S. 4,1 S 4,2 S 4,3 express; The output of the first convolutional kernel is connected to the input of the first ReLU activation function, the output of the first ReLU activation function is connected to the input of the second convolutional kernel, and the output of the second convolutional kernel is connected to the input of the second ReLU activation function, which is used to change the number of channels in the image feature map; the output of the second ReLU activation function is connected to the input of the first sub-network, which is used to extract image features; the first sub-network, the second sub-network, the third sub-network, and the fourth sub-network are arranged from left to right in sequence. The first subnet outputs feature map S out,1 The second subnet outputs feature map S out,2 The third subnet outputs feature map S out,3 The fourth subnet outputs feature map S out,4 The feature map S out,4 After performing one bilinear interpolation upsampling operation, the resolution of the feature map is increased to match that of feature map S. out,3 Consistent with and consistent with feature map S out,3 Element stacking is performed to form feature map F out,3 ; in the feature map F out,3 The feature map F is then obtained by performing convolution operations with the third and fourth convolution kernels. out,up3 ; the feature map F out,up3 After performing one bilinear interpolation upsampling operation, the resolution of the feature map is increased to match that of feature map S. out,2 Consistent with and consistent with feature map S out,2 The feature map F is obtained by stacking elements. out,2 ; in the feature map F out,2 Then, the feature map F is obtained by performing convolution operations with the fifth and sixth convolution kernels. out,up2 ; the feature map F out,up2 After performing one bilinear interpolation upsampling operation, the resolution of the feature map is increased to match that of feature map S. out,1 Consistent with and consistent with feature map S out,1 The feature map F is obtained by stacking elements. out,1 ; in the feature map F out,1 The feature map F is then obtained by performing convolution operations with the seventh and eighth convolution kernels. out,up1 ; The feature map S out,4 The feature map F is obtained by sending it into the first difference module. diff,4 The feature map F out,up3 The feature map F is obtained by performing a convolution operation on the output of the second difference module after inputting it into the ninth convolution kernel. f,up3 ; the feature map F diff,4 After performing one bilinear interpolation upsampling operation, the resolution of the feature map is increased to be comparable to that of feature map F. f,up3 Consistent with and consistent with feature map F f,up3 The feature map F is obtained by adding elements together. p,3 ; in the feature map F out,up2 The feature map F is obtained by performing a convolution operation on the output of the third difference module after inputting it into the tenth convolution kernel. f,up2 ; in the feature map F p,3 After performing one bilinear interpolation upsampling operation, the resolution of the feature map is increased to be comparable to that of feature map F. f,up2 Consistent with and consistent with feature map F f,up2 The feature map F is obtained by adding elements together. p,2 ; in the feature map F out,up1 After inputting the fourth difference module, the output undergoes an eleventh convolution operation to obtain the feature map F. f,up1 ; in the feature map F p,2 After performing one bilinear interpolation upsampling, the resolution of the feature map is increased to be comparable to that of feature map F. f,up1 Consistent with and consistent with feature map F f,up1 The feature map F is obtained by adding elements together. p,1 ; in the feature map F p,1 Then, the twelfth convolution kernel operation is performed to obtain the feature map F. out ; Step S20: Obtain labeled sample images to construct a dataset, and use the sample images in the dataset to train the modular learning network; Step S30: Input the image to be tested into the trained modular learning network. The modular learning network outputs the probability value corresponding to the region. If the probability value is greater than or equal to the threshold, the region is marked as a changed region; otherwise, it is marked as an unchanged region.

2. The remote sensing image change detection method based on modular learning networks as described in claim 1, characterized in that: In step S20, the process of training the modular learning network includes: Step S21: Obtain sample images from the dataset, adjust the sample images to RGB images of size 256×256, and then input them into the constructed modular learning network. Step S22: The modular learning network outputs feature maps F. out ; Step S23: Calculate the modular learning network using the binary classification cross-quotient loss function and the true labels. Loss value: ； in This represents all pixels in the sample image; Represents the actual label value. , This represents the probability value of the region of change predicted by the modular learning network. ; Calculate the Dice loss value of the modular learning network using the Dice loss function and the true labels: ; in This represents all pixels in the sample image; Represents the actual label value. , This represents the probability value predicted by the network. , Indicates a constant; Weighted The total loss of the modular learning network is obtained from the loss value and the Dice loss value. : in , ; Step S24: Based on the total loss The parameters of the modular learning network are updated backward using gradient descent until the total loss is reached. Training ends when the preset threshold is reached.

3. The remote sensing image change detection method based on modular learning networks as described in claim 1, characterized in that: The steps in step S30 include: After adjusting the image to be tested to a 256×256 RGB image, it is input into the trained modular learning network. Calculate the probability value of regions in an RGB image; mark regions with a probability value greater than or equal to 0.5 as changed regions, and mark regions with a probability value less than 0.5 as unchanged regions.

Images