A Remote Sensing Image Change Detection Method and System Based on Progressive Distraction Mining

Abstract

The present invention relates to a remote sensing image change detection method and system based on progressive distraction mining, belonging to the technical field of dual-temporal remote sensing image processing. The multi-scale features of dual-temporal remote sensing images are respectively extracted by a convolutional neural network. The absolute values of the differences between the dual-temporal features of the same scale are obtained to obtain multi-scale difference feature maps. The change region prediction map is obtained by using the obtained small-scale difference features, and the large-scale difference feature maps are guided to distract and mine the change regions according to the change region prediction map. Subsequently, the change region prediction map is obtained by using the guided and refined large-scale difference features to guide the refinement of the larger-scale difference features, so as to achieve progressive guidance. Finally, the obtained multi-scale fusion difference feature map is input into a predictor for prediction, and the loss value is calculated to update the model. The method of the present invention performs secondary mining on uncertain and fuzzy change regions, effectively avoiding the problems of false changes and missed detections.

Description

Technical Field

[0001] The invention relates to a remote sensing image change detection method, more precisely to a remote sensing image change detection method and system based on progressive distraction mining, belonging to the technical field of dual-temporal remote sensing image processing. Background Art

[0002] In recent years, aerospace technology has developed rapidly, and a large number of remote sensing satellites have been launched. With this comes the ability to obtain massive remote sensing images. Thanks to these remote sensing images, many large remote sensing image datasets have been established, meeting the data needs of remote sensing image-related tasks based on deep learning, and have made remarkable progress in urban change analysis, natural disaster prevention, agricultural damage assessment, etc.

[0003] Remote sensing image change detection is an important remote sensing task, which monitors the changes of surface objects by using remote sensing dual-phase images collected by remote sensing satellites. With the development of deep learning, many deep learning-based methods have been used for remote sensing change detection. For example, convolutional neural networks, thanks to their significant advantages in extracting detailed features, are used by many methods to extract the features of dual-phase images separately, and then fuse the obtained dual-phase features to identify the location of the change area. However, remote sensing images are easily affected by environmental factors such as lighting and atmosphere and become blurred, which in turn increases the uncertainty and ambiguity of the change area. At present, deep learning-based methods have encountered difficulties in dealing with these blurred change areas, which may cause pseudo changes or missed changes, which poses a major challenge to remote sensing change detection methods based on deep learning. Summary of the invention

[0004] The purpose of the present invention is to overcome the above shortcomings and provide a remote sensing image change detection method based on progressive distraction mining.

[0005] This method performs secondary mining on the uncertain fuzzy change area, effectively avoiding the problems of false changes and missed changes.

[0006] The technical solution adopted by the present invention is:

[0007] A remote sensing image change detection method based on progressive distraction mining includes the following steps:

[0008] S1. Obtain original remote sensing images, preprocess and divide the data set;

[0009] S2. Input two remote sensing images of different phases into the weight-sharing twin feature extractor to obtain multiple feature maps of different scales;

[0010] S3. taking the difference of the obtained dual-phase features of the same scale and calculating the absolute value to obtain a multi-scale difference feature map;

[0011] S4. Progressive guiding strategy for distraction mining: First, use the obtained minimum-scale difference features to predict the change region prediction map. Use the change region prediction map of the minimum scale to guide the decoding of the large-scale difference feature map one level larger than it, and focus on predicting the change region and the non-change region respectively for distraction mining. Then, connect the outputs of the two branches of distraction mining along the channels to obtain a preliminary refined difference feature map; The preliminary refined difference feature map is used as the small-scale difference feature to obtain the change region prediction map, and then guide the decoding of the large-scale difference feature map one level larger than it, and repeat the distraction mining until a refined multi-scale fusion difference feature map output by distraction mining and channel splicing of the largest-scale difference feature map is obtained;

[0012] S5. Upsample the refined multi-scale fusion difference feature map and send it to the predictor to output the final prediction map. Compare the prediction result with the image patch label, use the binary cross-entropy function as the loss function, calculate the loss value, and perform gradient backpropagation to update the model parameters;

[0013] S6. Conduct validation during the training process. Finally, select the model with the best performance on the test set as the final network model, and input the remote sensing image into this model for change detection.

[0014] In the above method, the Siamese feature extractor described in step S2 uses the Siamese ResNet-50, and downsampling is used in the feature extractor to reduce the feature scale.

[0015] The distraction mining described in step S4 is to make a prediction on the small-scale difference feature map to obtain the change region prediction map. After upsampling, the change region prediction Figure 1 path is multiplied element-wise with the large-scale difference feature map one level larger than it to enhance the attention of the difference feature map to the change region, and use 3 ×3 convolution to further mine the details of the change region and remove the false changes. The other path uses (1 - P4 U That is, subtract P4 from a matrix full of 1s U to obtain the uncertain change region) multiplied by the large-scale difference feature map one level larger than it to enhance the attention of the difference feature map to the uncertain change region, and use 3 ×3 convolution to further mine the details of the uncertain change region.

[0016] The expression of the loss function described in step S5 is as follows:

[0017] ,

[0018] Wherein, N is the number of pixels in each image patch; M is the number of categories, with a value of 2 here, i represents the i-th pixel point in the image patch, and its value range is [1, N]; j represents the j-th category, and its value range is [1, M]; y ij is the sign function, which takes 1 if the true category of pixel point i is equal to j, otherwise it takes 0; p ij is the probability value that pixel point i belongs to category j.

[0019] Another object of the present invention is to provide a remote sensing image change detection system based on progressive distraction mining, including a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, it implements the remote sensing image change detection method based on progressive distraction mining as described above.

[0020] The beneficial effects of the present invention are as follows:

[0021] Through the distraction mining module, the present invention uses the small-scale change prediction map to guide the feature extraction of the large-scale difference feature map, thereby enhancing the detail extraction of the change area in the large-scale difference feature map. At the same time, the reverse small-scale change prediction map is used to enhance the feature extraction of the fuzzy change area in the large-scale difference feature map. The progressive method enables the final difference feature map to fuse rich semantic and detail information. The present invention effectively extracts the details of the change area, mines the features of the uncertain change area, overcomes the deficiencies of the existing remote sensing image change detection methods in the face of uncertain change areas, reduces the situations of false changes and missed detection of change areas, and finally obtains a remote sensing change detection model with higher accuracy. Description of the Drawings

[0022] Figure 1 is the flowchart of the method of the present invention;

[0023] Figure 2 is the network structure diagram of the method model of the present invention. Detailed Embodiments

[0024] The present invention will be further described below in conjunction with specific embodiments.

[0025] Embodiment 1 A remote sensing image change detection method based on progressive distraction mining, including the following steps:

[0026] S1. Obtain the original remote sensing image, preprocess it and divide the data set:

[0027] Divide the original remote sensing image data set into a training set, a validation set and a test set according to the ratio of 6:2:2. Train the model on the training set, use the model with the best result on the validation set as the final convergent model, and evaluate the model effect on the test set.

[0028] High-resolution remote sensing images have large sizes. Limited by computing resources, the original dual-temporal images need to be cropped into 256 image patches of 256 before being fed into the model, where the dual-temporal image patches are pairwise corresponding. Subsequently, we perform data augmentation. First, we calculate the mean and standard deviation of the RGB channels of all image patches and standardize the pixel values of the image patches. Second, we perform data augmentation operations such as the same horizontal and vertical random flips on the corresponding pair of dual-temporal image patches and swapping the time order of the pair of dual-temporal image patches.

[0029] S2. Input two remote sensing images of different time phases into a weight-sharing Siamese feature extractor to obtain multiple feature maps of different scales respectively:

[0030] Select the weight-sharing Siamese Residual Network ResNet-50 as the feature extractor for remote sensing dual-temporal images, which can extract four-scale feature information. In the feature extractor of the present invention, downsampling is used to reduce the feature scale. ResNet-50 contains residual structures, enabling the network to extract deep features. Taking a pair of dual-temporal image patches of size 256 256 as an example, input this pair of image patches into the weight-sharing Siamese feature extractor. The image of time phase 1 and the image of time phase 2 respectively obtain four feature maps of different scales, with sizes of 64 64 256, 32 32 512, 16 16 1024, and 8 8 2048, where the last number represents the number of channels of the current feature map. Denote the four feature maps of the time phase 1 image as 、 、 、 , and denote the four feature maps of the time phase 2 image as 、 、 、 .

[0031] S3. Obtain multi-scale difference feature maps by taking the absolute value of the difference between the obtained dual-temporal features of the same scale:

[0032] Subtract the dual-temporal features of the same scale and take the absolute value to obtain multi-scale difference feature maps. Taking a pair of dual-temporal feature maps of the same scale and as an example, obtain D1 through Abs( ), with a size of 64 64 256, where Abs is the absolute value operation. Further, the present invention can obtain D2, D3, and D4, with sizes of 32 32 512, 16 16 1024, and 8 8 2048.

[0033] S4. Use the progressive guidance strategy for distraction mining:

[0034] In small-scale features, one pixel represents multiple pixels in large-scale features. Features with smaller scales belong to deep semantic features, which have a smaller resolution but contain rich semantic information. Features with larger scales belong to shallow detail features, which have a large resolution and contain rich spatial details. Therefore, the present invention uses the method of progressive distraction mining to gradually distract and mine the changing area, reducing the uncertainty of the changing area and thus avoiding the situations of false changes and missed detections.

[0035] The present invention progressively distracts and mines the multi-scale difference features through the distraction mining module, that is, uses the changes predicted by the smaller scale to guide the decoding of the larger scale difference feature maps. First, use the obtained smallest scale difference feature to predict the change area prediction map, and use the change area prediction map of the smallest scale to guide the decoding of the larger scale difference feature map one level larger than it, respectively focusing on predicting the changing area and the non-changing area for distraction mining, and then connect the outputs of the two branches of distraction mining along the channel to obtain a preliminary refined difference feature map; the preliminary refined difference feature map is used as the small-scale difference feature to obtain the change area prediction map, and then guide the decoding of the larger scale difference feature map one level larger than it, repeating the distraction mining until the refined multi-scale fusion difference feature map output by distraction mining and channel splicing of the largest scale difference feature map is obtained.

[0036] Taking the difference feature maps with scales of 8 8 2048 and scale 16 16 1024 as an example, the specific method is to let the smaller scale difference feature map D4 make a prediction to get P4∈[0,1], and then use bilinear upsampling by a factor of two to upsample P4 to get P4 U , with a scale of 16 16 1. Subsequently, use the element-wise multiplication of P4 U and D3 to guide the decoding of D3. This guiding method helps to enhance D3's attention to the changing area, and use 3 3 convolution to further mine the details of the changing area and remove false changes. At the same time, use (1 - P4 U) Multiply by D3 to enhance D3's attention to the uncertain change regions, and utilize 3 3 convolutions to further mine the overlooked change information in the uncertain change regions to reduce the missed detected change regions. This guidance helps the differential features to focus on and deeply mine some uncertain change regions, thereby reducing the situation of missed detection. This guidance and mining method is called distraction mining. The present invention can mine more change details and avoid the situations of pseudo-changes and missed detected change regions. Then, the refined differential feature map D3 is obtained by connecting the outputs of the two branches of distraction mining along the channels R . This is the process of a distraction mining module (DMM).

[0037] Subsequently, D3 R is used for prediction to obtain P3 ∈ [0, 1]. P3 is sent to the next DMM module and guides the distraction mining of D2 to obtain D2 R . Subsequently, D2 R is used for prediction to obtain P2 ∈ [0, 1]. P2 is sent to the next DMM module and guides the distraction mining of D1, and finally D1 with a scale of 64 64 × 256 is obtained R . It should be noted that the distraction mining process of the present invention is progressive. D1 R fuses the differential feature maps of three scales of D2, D3, and D4. Therefore, D1 R contains both a large amount of spatial details of the change regions and rich semantic information of the change regions.

[0038] S5. After upsampling the refined multi-scale fusion differential feature map, it is sent to the predictor to output the final prediction map. The prediction result is compared with the image patch label. Using the binary cross-entropy function as the loss function, calculate the loss value, and perform gradient backpropagation to update the model parameters:

[0039] Taking the multi-scale fusion differential feature map D1 R as the final differential feature map, after upsampling, it is sent to the predictor. The predictor includes two convolutional layers with a convolutional kernel size of 3 × 3 and a Sigmoid function. This predictor can output the final prediction map. Specifically, D1 R obtains a final prediction map with a scale of 256 × 256 × 1 after passing through this convolutional layer. The present invention believes that the positions with values greater than 0.5 in this prediction map belong to the change regions, and the positions with values less than 0.5 belong to the non-change regions. The prediction result is compared with the image patch label. Using the binary cross-entropy function as the loss function, calculate the loss value, and perform gradient backpropagation to update the model parameters. The expression of the cross-entropy loss function is as follows:

[0040] ,

[0041] where N is the number of pixels in each image patch; M is the number of classes, with a value of 2 here, i represents the i-th pixel point in the image patch, and its value range is [1, N]; j represents the j-th class, and its value range is [1, M]; y ij is the sign function, which takes 1 if the true class of pixel point i is equal to j, otherwise 0; p ij is the probability value that pixel point i belongs to class j.

[0042] S6. Validation is carried out simultaneously during the training process, and finally the model with the best performance on the test set is selected as the final network model, and the remote sensing image is input into this model for change detection:

[0043] Within the framework of Pytorch, a NVIDIA GeForce RTX 3090 GPU is used for training, and the Adam optimizer with a momentum of 0.9 and a weight decay coefficient of 0.0001 ( ) is used to optimize the network. In the present invention, the learning rate is set to 1×10 -4 , trained for 100 epochs, the batch training size is set to 32, and the learning rate decreases linearly as the epoch increases. In the present invention, validation is carried out simultaneously during the training process, and finally the model with the best performance on the test set is selected as the final network model.

[0044] Embodiment 2 A remote sensing image change detection system based on progressive distraction mining, including a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, it implements the remote sensing image change detection method based on progressive distraction mining as described in Embodiment 1 above.

[0045] The above is a further description of the present invention in combination with embodiments, and the protection scope of the present invention is not limited thereto.

Claims

1. A remote sensing image change detection method based on progressive distraction mining, characterized in that: The steps include: S1. Obtain original remote sensing images, preprocess and divide the data set; S2. Input two remote sensing images of different phases into the weight-sharing twin feature extractor to obtain multiple feature maps of different scales; S3. taking the difference of the obtained dual-phase features of the same scale and calculating the absolute value to obtain a multi-scale difference feature map; S4. Use progressive guidance strategy for distracted mining: First, use the obtained minimum-scale difference feature prediction to obtain the change region prediction map, use the minimum-scale change region prediction map to guide the decoding of the large-scale difference feature map one level larger than it, focus on predicting the change region and predicting the non-change region for distracted mining respectively, and then connect the outputs of the two branches of distracted mining along the channel to obtain a preliminary refined difference feature map; the preliminary refined difference feature map is used as a small-scale difference feature to obtain the change region prediction map, and then guide the decoding of the large-scale difference feature map one level larger than it, and repeat the distracted mining until the refined multi-scale fusion difference feature map output by distracted mining of the maximum-scale difference feature map and channel splicing is obtained; S5. The refined multi-scale fusion difference feature map is upsampled and sent to the predictor to output the final prediction map. The prediction result is compared with the image block label, and the binary cross entropy function is used as the loss function to calculate the loss value and perform gradient back propagation to update the model parameters. S6. Verification is performed during the training process, and finally the model with the best effect on the test set is selected as the final network model. The remote sensing image is input into this model for change detection.

2. According to claim 1, a remote sensing image change detection method based on progressive distraction mining is characterized in that: The twin feature extractor described in step S2 uses the twin residual network ResNet-50, and the feature extractor uses downsampling to reduce the feature scale.

3. The remote sensing image change detection method based on progressive distraction mining according to claim 1 is characterized in that: The distraction mining described in step S4 is to predict the small-scale difference feature map to obtain the change area prediction map. After upsampling, the change area prediction map is multiplied element by element with the large-scale difference feature map of the next level to enhance the difference feature map's attention to the change area, and use 3 3 convolutions further explore the details of the changed area and remove pseudo changes, and the other one uses 1-P4 U Multiply it with the large-scale difference feature map of the next level to enhance the attention of the difference feature map to the uncertain change area, and use 3 3 convolutions further explore the details of uncertain change areas.

4. The remote sensing image change detection method based on progressive distraction mining according to claim 1 is characterized in that: The expression of the loss function described in step S5 is as follows: ， Where N is the number of pixels in each image block; M is the number of categories, which is 2 here; i represents the i-th pixel in the image block, and its value range is [1, N]; j represents the j-th category, and its value range is [1, M]; y ij is a sign function, which takes the value 1 if the true category of pixel i is equal to j, otherwise it takes the value 0; p ij is the probability value that pixel i belongs to category j.

5. A remote sensing image change detection system based on progressive distraction mining, comprising a processor and a memory storing computer program instructions; wherein: When the processor executes the computer program instructions, the remote sensing image change detection method based on progressive distraction mining as described in any one of claims 1 to 4 is implemented.

Images