Remote sensing image change detection method and system based on progressive distraction mining
By using a progressive distraction mining method, multi-scale features are extracted and distraction mining is performed using a twin residual network, which solves the problems of false changes and missed detections in remote sensing image change detection and achieves higher accuracy in change region detection.
Patent Information
- Application Number
- CN202411146627.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-21
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2044-08-21
AI Technical Summary
Existing deep learning-based remote sensing image change detection methods are prone to false changes or missed changes when dealing with blurred change areas, making it difficult to effectively distinguish uncertain change areas.
A progressive distraction mining method is adopted, which extracts multi-scale feature maps through a weight-sharing Siamese residual network and uses a progressive guidance strategy to gradually enhance the focus on changing regions. The model parameters are optimized by combining a binary classification cross-entropy function to achieve refined extraction of uncertain changing regions.
It effectively reduces false changes and missed changes, improves the accuracy of remote sensing image change detection, enhances the extraction of details and semantic information of changed areas, and improves the accuracy of the detection model.
Smart Images

Figure CN118968340A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a remote sensing image change detection method, more precisely to a remote sensing image change detection method and system based on progressive distraction mining, belonging to the technical field of dual-temporal remote sensing image processing. Background Art
[0002] In recent years, aerospace technology has developed rapidly, and a large number of remote sensing satellites have been launched. With this comes the ability to obtain massive remote sensing images. Thanks to these remote sensing images, many large remote sensing image datasets have been established, meeting the data needs of remote sensing image-related tasks based on deep learning, and have made remarkable progress in urban change analysis, natural disaster prevention, agricultural damage assessment, etc.
[0003] Remote sensing image change detection is an important remote sensing task, which monitors the changes of surface objects by using remote sensing dual-phase images collected by remote sensing satellites. With the development of deep learning, many deep learning-based methods have been used for remote sensing change detection. For example, convolutional neural networks, thanks to their significant advantages in extracting detailed features, are used by many methods to extract the features of dual-phase images separately, and then fuse the obtained dual-phase features to identify the location of the change area. However, remote sensing images are easily affected by environmental factors such as lighting and atmosphere and become blurred, which in turn increases the uncertainty and ambiguity of the change area. At present, deep learning-based methods have encountered difficulties in dealing with these blurred change areas, which may cause pseudo changes or missed changes, which poses a major challenge to remote sensing change detection methods based on deep learning. Summary of the invention
[0004] The purpose of the present invention is to overcome the above shortcomings and provide a remote sensing image change detection method based on progressive distraction mining. This method performs secondary mining on the uncertain fuzzy change area, effectively avoiding the problems of false changes and missed changes.
[0005] The technical solution adopted by the present invention is: A remote sensing image change detection method based on progressive distraction mining includes the following steps: S1. Obtain original remote sensing images, preprocess and divide the data set; S2. Input two remote sensing images of different phases into the weight-sharing twin feature extractor to obtain multiple feature maps of different scales; S3. taking the difference of the obtained dual-phase features of the same scale and calculating the absolute value to obtain a multi-scale difference feature map; S4. Use progressive guidance strategy for distracted mining: First, use the obtained minimum-scale difference feature prediction to obtain the change region prediction map, use the minimum-scale change region prediction map to guide the decoding of the large-scale difference feature map one level larger than it, focus on predicting the change region and predicting the non-change region for distracted mining respectively, and then connect the outputs of the two branches of distracted mining along the channel to obtain a preliminary refined difference feature map; the preliminary refined difference feature map is used as a small-scale difference feature to obtain the change region prediction map, and then guide the decoding of the large-scale difference feature map one level larger than it, and repeat the distracted mining until the refined multi-scale fusion difference feature map output by distracted mining of the maximum-scale difference feature map and channel splicing is obtained; S5. The refined multi-scale fusion difference feature map is upsampled and sent to the predictor to output the final prediction map. The prediction result is compared with the image block label, and the binary cross entropy function is used as the loss function to calculate the loss value and perform gradient back propagation to update the model parameters. S6. Verification is performed during the training process, and finally the model with the best effect on the test set is selected as the final network model. The remote sensing image is input into this model for change detection.
[0006] In the above method, the twin feature extractor described in step S2 uses the twin residual network ResNet-50, and the feature extractor uses downsampling to reduce the feature scale.
[0007] The distraction mining described in step S4 is to predict the small-scale difference feature map to obtain the change area prediction map. After upsampling, the change area prediction Figure 1 The large-scale difference feature map of the next level is multiplied element by element to enhance the attention of the difference feature map to the change area, and 3 3 convolutions further explore the details of the changed area and remove pseudo changes, and the other one uses (1-P4 U , that is, subtract P4 from a matrix full of 1s U The uncertain change area is obtained by multiplying it with the large-scale difference feature map of a larger level to enhance the attention of the difference feature map to the uncertain change area, and using 3 3 convolutions further explore the details of uncertain change areas.
[0008] The expression of the loss function described in step S5 is as follows: , Where N is the number of pixels in each image block; M is the number of categories, which is 2 here; i represents the i-th pixel in the image block, and its value range is [1, N]; j represents the j-th category, and its value range is [1, M]; y ij is a sign function, which takes the value 1 if the true category of pixel i is equal to j, otherwise it takes the value 0;p ij is the probability value that pixel i belongs to category j.
[0009] Another object of the present invention is to provide a remote sensing image change detection system based on progressive distraction mining, comprising a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, it implements the remote sensing image change detection method based on progressive distraction mining as described above.
[0010] The beneficial effects of the present invention are: The present invention uses a distraction mining module and a small-scale change prediction map to guide the feature extraction of a larger-scale difference feature map, thereby enhancing the detail extraction of the change area in the larger difference feature map. At the same time, the reverse small-scale change prediction map is used to enhance the feature extraction of the fuzzy change area in the larger difference feature map. The final difference feature map is integrated with rich semantic and detail information in a progressive manner. The present invention effectively extracts the details of the change area and mines the features of the uncertain change area, overcomes the shortcomings of the existing remote sensing image change detection method when facing uncertain change areas, reduces the situation of pseudo-changes and missed change areas, and finally obtains a more accurate remote sensing change detection model. BRIEF DESCRIPTION OF THE DRAWINGS
[0011] Figure 1 is a flow chart of the method of the present invention; Figure 2 This is a network structure diagram of the model of the method of the present invention. DETAILED DESCRIPTION
[0012] The present invention is further described below with reference to specific embodiments.
[0013] Embodiment 1 A remote sensing image change detection method based on progressive distraction mining comprises the following steps: S1. Obtain the original remote sensing image, preprocess it and divide the data set: The original remote sensing image dataset is divided into training set, validation set and test set in a ratio of 6:2:2. The model is trained on the training set, and the model that achieves the best result on the validation set is taken as the final converged model. The model effect is evaluated on the test set.
[0014] High-resolution remote sensing images are very large in size. Due to limited computing resources, the original dual-phase images need to be cropped into 256 The 256-bit image blocks are then fed into the model, where the two-phase image blocks correspond to each other. Then we perform data augmentation. First, we calculate the mean and standard deviation of the three RGB channels of all image blocks and standardize the pixel values of the image blocks. Then, we perform the same data augmentation operations on the corresponding pair of two-phase image blocks, such as horizontal and vertical random flipping and swapping the time order of the two-phase image blocks.
[0015] S2. Input two remote sensing images of different phases into the weight-sharing twin feature extractor to obtain multiple feature maps of different scales: The weight-sharing twin residual network ResNet-50 is selected as the feature extractor of remote sensing dual-phase images, which can extract feature information at four scales. In the feature extractor of the present invention, downsampling is used to reduce the feature scale. ResNet-50 contains a residual structure, which enables the network to extract deep features. Taking a pair of dual-phase image blocks of size 256 as an example, the pair of image blocks are input into the weight-sharing twin feature extractor, and four feature maps of different scales are obtained for the phase 1 image and the phase 2 image, respectively, with sizes of 64 64 256, 32 32 512, 16 16 1024 and 8 8 2048, where the last number represents the number of channels of the current feature map. The four feature maps of the phase 1 image are recorded as , , , , the four feature maps of the phase 2 image are recorded as , , , .
[0016] S3. Take the difference of the obtained dual-phase features of the same scale and calculate the absolute value to obtain the multi-scale difference feature map: Subtract the bi-phase features of the same scale and take the absolute value to obtain a multi-scale difference feature map. and For example, through Abs ( ) Get D1, size 64 64 256, where Abs is an absolute value operation. Further, the present invention can obtain D2, D3 and D4, the sizes of which are 32 respectively. 32 512, 16 16 1024 and 8 8 2048.
[0017] S4. Use progressive guidance strategy to distract mining: In small-scale features, one pixel represents multiple pixels in large scale. Small-scale features belong to deep semantic features, which have low resolution but contain rich semantic information. Large-scale features belong to shallow detail features, which have high resolution and contain rich spatial details. Therefore, the present invention uses a progressive distraction mining method to distractively mine the change area step by step, reducing the uncertainty of the change area and avoiding pseudo changes and missed changes.
[0018] The present invention uses a distraction mining module to progressively distraction mine multi-scale difference features, that is, to use smaller-scale change predictions to guide the decoding of larger-scale difference feature maps. First, the minimum-scale difference feature prediction is used to obtain a change region prediction map, and the minimum-scale change region prediction map is used to guide the decoding of a larger-scale difference feature map, focusing on predicting change regions and predicting non-change regions for distraction mining, respectively, and then connecting the outputs of the two branches of distraction mining along the channel to obtain a preliminary refined difference feature map; the preliminary refined difference feature map is used as a small-scale difference feature to obtain a change region prediction map, and then guides the decoding of a larger-scale difference feature map, and repeats distraction mining until a refined multi-scale fusion difference feature map is obtained, which is output by distraction mining of the maximum-scale difference feature map and channel splicing.
[0019] With scale 8 8 2048 and scale 16 16 Taking the 1024 difference feature map as an example, the specific method is to make the smaller scale difference feature map D4 predict to get P4∈[0,1], and then use two times linear upsampling to upsample P4 to get P4 U , whose scale is 16 16 1. Then P4 U The D3 decoding is guided by multiplying the D3 elements one by one. This guidance helps to enhance D3's attention to the changing area and utilize 3 3 convolutions to further explore the details of the changed area and remove pseudo changes. At the same time, using (1-P4 U ) is multiplied with D3 to enhance D3's attention to the uncertain change area, and 3 3. Convolution further mines the ignored change information in the uncertain change area to reduce the missed change area. This guidance helps the difference feature focus on and deeply mine some uncertain change areas, thereby reducing the missed detection. We call this guided mining method distraction mining. The present invention can mine more change details and avoid pseudo changes and missed change areas. Then, the refined difference feature map D3 is obtained by connecting the outputs of the two branches of distraction mining along the channel. R This is the process of a distraction mining module (DMM).
[0020] Then D3 R Make a prediction to get P3∈[0,1], send P3 to the next DMM module and guide the distraction mining of D2 to get D2 R Then D2 R Make a prediction and get P2∈[0,1], send P2 to the next DMM module and guide the distraction mining of D1, and finally get a scale of 64 64 256 D1 R It is worth noting that the distraction mining process of the present invention is progressive. R The difference feature maps of three scales D2, D3 and D4 are integrated, so D1 R It contains not only a large amount of spatial details of the change region but also rich semantic information of the change region.
[0021] S5. The refined multi-scale fusion difference feature map is upsampled and sent to the predictor to output the final prediction map. The prediction result is compared with the image block label. The binary cross entropy function is used as the loss function to calculate the loss value and perform gradient backpropagation to update the model parameters: The multi-scale fusion difference feature map D1 R As the final difference feature map, it is sent to the predictor after upsampling. The predictor contains two convolution kernels with a size of 3 3 convolutional layers and a Sigmoid function, the predictor can output the final prediction map. Specifically, D1 R After the convolution layer, a scale of 256 is obtained. 256 1, the present invention believes that the positions where the value is greater than 0.5 in this prediction map belong to the change area, and the positions where the value is less than 0.5 belong to the non-change area. The prediction result is compared with the image block label, and the binary cross entropy function is used as the loss function to calculate the loss value, and gradient back propagation is performed to update the model parameters. The expression of the cross entropy loss function is as follows: , Where N is the number of pixels in each image block; M is the number of categories, which is 2 here; i represents the i-th pixel in the image block, and its value range is [1, N]; j represents the j-th category, and its value range is [1, M]; y ij is a sign function, which takes the value 1 if the true category of pixel i is equal to j, otherwise it takes the value 0; p ij is the probability value that pixel i belongs to category j.
[0022] S6. Verify during the training process, and finally select the model with the best effect on the test set as the final network model. Input the remote sensing image into this model for change detection: In the Pytorch framework, an NVIDIA GeForce RTX 3090 GPU was used for training, using the Adam optimizer with momentum of 0.9 and weight decay factor of 0.0001 ( ) to optimize the network. The present invention sets the learning rate to 1×10 -4 , train for 100 epochs, set the batch training size to 32, and the learning rate decreases linearly with the increase of epochs. The present invention performs verification during the training process, and finally selects the model with the best effect on the test set as the final network model.
[0023] Embodiment 2 A remote sensing image change detection system based on progressive distraction mining comprises a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the remote sensing image change detection method based on progressive distraction mining as described in Embodiment 1 above is implemented.
[0024] The above is a further description of the present invention in combination with the embodiments, and the protection scope of the present invention is not limited thereto.
Claims
1. A remote sensing image change detection method based on progressive distraction mining, characterized in that: The steps include: S1. Obtain original remote sensing images, preprocess and divide the data set; S2. Input two remote sensing images of different phases into the weight-sharing twin feature extractor to obtain multiple feature maps of different scales; S3. taking the difference of the obtained dual-phase features of the same scale and calculating the absolute value to obtain a multi-scale difference feature map; S4. Use progressive guidance strategy for distracted mining: First, use the obtained minimum-scale difference feature prediction to obtain the change region prediction map, use the minimum-scale change region prediction map to guide the decoding of the large-scale difference feature map one level larger than it, focus on predicting the change region and predicting the non-change region for distracted mining respectively, and then connect the outputs of the two branches of distracted mining along the channel to obtain a preliminary refined difference feature map; the preliminary refined difference feature map is used as a small-scale difference feature to obtain the change region prediction map, and then guide the decoding of the large-scale difference feature map one level larger than it, and repeat the distracted mining until the refined multi-scale fusion difference feature map output by distracted mining of the maximum-scale difference feature map and channel splicing is obtained; S5. The refined multi-scale fusion difference feature map is upsampled and sent to the predictor to output the final prediction map. The prediction result is compared with the image block label, and the binary cross entropy function is used as the loss function to calculate the loss value and perform gradient back propagation to update the model parameters. S6. Verification is performed during the training process, and finally the model with the best effect on the test set is selected as the final network model. The remote sensing image is input into this model for change detection.
2. According to claim 1, a remote sensing image change detection method based on progressive distraction mining is characterized in that: The twin feature extractor described in step S2 uses the twin residual network ResNet-50, and the feature extractor uses downsampling to reduce the feature scale.
3. The remote sensing image change detection method based on progressive distraction mining according to claim 1 is characterized in that: The distraction mining described in step S4 is to predict the small-scale difference feature map to obtain the change area prediction map. After upsampling, the change area prediction map is multiplied element by element with the large-scale difference feature map of the next level to enhance the difference feature map's attention to the change area, and use 3 3 convolutions further explore the details of the changed area and remove pseudo changes, and the other one uses 1-P4 U Multiply it with the large-scale difference feature map of the next level to enhance the attention of the difference feature map to the uncertain change area, and use 3 3 convolutions further explore the details of uncertain change areas.
4. The remote sensing image change detection method based on progressive distraction mining according to claim 1 is characterized in that: The expression of the loss function described in step S5 is as follows: , Where N is the number of pixels in each image block; M is the number of categories, which is 2 here; i represents the i-th pixel in the image block, and its value range is [1, N]; j represents the j-th category, and its value range is [1, M]; y ij is a sign function, which takes the value 1 if the true category of pixel i is equal to j, otherwise it takes the value 0; p ij is the probability value that pixel i belongs to category j.
5. A remote sensing image change detection system based on progressive distraction mining, comprising a processor and a memory storing computer program instructions; wherein: When the processor executes the computer program instructions, the remote sensing image change detection method based on progressive distraction mining as described in any one of claims 1 to 4 is implemented.
Citation Information
Patent Citations
Image anomaly detection method based on core memory bank residual error
CN117079029A
Remote sensing change detection method based on multi-scale feature mining and reflection mechanism
CN118334488A
Method and system of extraction of impervious surface of remote sensing image
US20200026953A1
Cited By
Remote sensing image change detection method and system based on prior knowledge and frequency domain learning
CN120219983A
Remote sensing image change detection method and system based on style alignment and edge constraint
CN120339709A
