A Remote Sensing Image Change Detection Method and Device

Abstract

A remote sensing image change detection method and device provided by the present invention extract feature maps of different time phases and the fusion change features of feature maps of different time phases, and use an unsupervised learning module to reduce the gap of the features of the changed areas in the feature maps of different time phases so as to strengthen the change features, thereby improving the accuracy of change detection.

Description

Technical Field

[0001] The present invention belongs to the technical field of satellite remote sensing, and in particular relates to a method and device for remote sensing image change detection. Background Art

[0002] Remote sensing image change detection is to detect the change information of surface features in the same geographical area at different times through image processing and mathematical models. Remote sensing image change detection has a wide range of applications in the fields of ecological environment monitoring, disaster emergency management, surface cover monitoring, etc. In recent years, the rapid development of deep learning technology has played an important role in the field of digital image recognition.

[0003] In the actual work of land law enforcement, it is still mainly based on the method of manual on-site investigation and evidence collection. If remote sensing images can be used for cultivated land change detection and new construction land on the cultivated land can be extracted therefrom, the human, time and financial costs can be greatly reduced. Summary of the Invention

[0004] The technical problem to be solved by the present invention is how to detect and analyze remote sensing images to obtain the changes between cultivated land and construction land in images at different times and improve the accuracy of change detection. A method and device for remote sensing image change detection are proposed.

[0005] To solve the above technical problem, the technical solution adopted by the present invention is:

[0006] A method for remote sensing image change detection includes the following steps:

[0007] Step 1: Extract features from two remote sensing images at time T1 and time T2 of the same area to obtain two feature maps F1 and F2 of different time phases;

[0008] Step 2: Stack the images at time T1 and time T2 in the channel dimension and then perform feature fusion to extract a fused change feature map F3;

[0009] Step 3: Input the two feature maps F1 and F2 of different time phases and the fused change feature map F3 into a neural network model for feature enhancement and fusion and binarization processing to obtain a change detection result.

[0010] Further, the method for extracting the two feature maps F1 and F2 of different time phases and the fused change feature map F3 is to use a backbone feature extraction network.

[0011] Further, the backbone feature extraction network is composed of a three-branch encoding-decoding network. The images at time T1 and time T2 are respectively input into the encoding-decoding networks of the upper and lower branches of the backbone feature extraction network for feature extraction to obtain two feature maps F1 and F2 of different time phases;

[0012] Stack the images at time T1 and the images at time T2 in the channel dimension, and then input them into the encoding-decoding network of the middle branch of the backbone feature extraction network for feature fusion and feature extraction to obtain the fused change feature map F3.

[0013] Furthermore, the neural network model used in step 3 is the change feature enhancement network CFI-Net. The network structure of the change feature enhancement network CFI-Net includes an unsupervised learning module for feature enhancement of the unchanged region features in two time phases and a feature fusion module for feature fusion of the difference feature F c and the fused change feature map F3. The difference feature F c refers to the difference between the feature map F1 extracted from the image at time T1 and the feature map F2 extracted from the image at time T2 for the two time phases.

[0014] Furthermore, the unsupervised learning module includes a Sigmoid classifier, a threshold converter, and two multipliers. Input the fused change feature F3 into the Sigmoid classifier to obtain the classification result, and input the classification result into the threshold converter for threshold conversion to obtain the unchanged change feature map M uc ;

[0015] Multiply the unchanged change feature map M uc separately with F1 and F2 to obtain the unchanged feature maps F uc1 and F uc2 at time T1 and time T2,

[0016] Calculate the loss L uc1 and F uc2 for the unchanged feature maps at time T1 and time T2, and minimize the loss L mse to reduce the gap between the unchanged feature maps F mse and F uc1 and F uc2 .

[0017] Furthermore, the feature fusion module for feature fusion of the difference feature F c and the fused change feature map F3 consists of two transposed convolutions with a kernel size of 3×3, a 1×1 convolution, and an Argmax function. Subtract the feature map F1 extracted from the image at time T1 from the feature map F2 extracted from the image at time T2 to obtain the difference feature F c between the two time phases. Stack the difference feature F c and the fused change feature map F3 in the channel dimension and input them into the feature fusion module for feature fusion and binarization processing to obtain the final binary image.

[0018] The present invention also provides a remote sensing image change detection device, including the following modules:

[0019] Feature extraction module for different time phases: used to extract features from two remote sensing images at time T1 and time T2 of the same area, obtaining two feature maps F1 and F2 of different time phases;

[0020] Fusion change feature map extraction module: used to stack the images at time T1 and time T2 in the channel dimension, then perform feature fusion and extract the fused change feature map F3;

[0021] Detection result output module: input the two feature maps F1 and F2 of different time phases and the fused change feature F3 into a neural network model for feature enhancement and fusion to obtain the change detection result.

[0022] Adopting the above technical solution, the present invention has the following beneficial effects:

[0023] A remote sensing image change detection method and device provided by the present invention, by extracting feature maps of different time phases and the fusion change features of feature maps of different time phases, uses an unsupervised learning module to reduce the gap of the features of the changed areas in the feature maps of different time phases to enhance the change features, thereby improving the accuracy of change detection. Brief Description of the Drawings

[0024] Figure 1 It is a schematic diagram of the remote sensing image change detection process of the present invention.

[0025] Figure 2 It is a schematic diagram of the CFI-Net structure;

[0026] Figure 3 It is an experimental result diagram of remote sensing images, where, Figure 3 (a) is the image at time T1, Figure 3 (b) is the image at time T2, Figure 3 (c) is the detection result image of the method proposed by the present invention, Figure 3 (d) is the reference image. Detailed Embodiment

[0027] The technical solution of the present invention will be clearly and completely described below with reference to the drawings. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of them. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0028] Figure 1 and Figure 2 shows a specific embodiment of a remote sensing image change detection method of the present invention, including the following steps:

[0029] Step 1: Extract features from two remote sensing images at time T1 and T2 in the same area to obtain two feature maps F1 and F2 at different time phases;

[0030] Step 2: Stack the images at time T1 and T2 in the channel dimension and then perform feature fusion to extract the fused change feature map F3;

[0031] In this embodiment, the method for extracting the two feature maps F1 and F2 at different time phases and the fused change feature map F3 is to use a backbone feature extraction network.

[0032] The backbone feature extraction network is composed of a three-branch encoding-decoding network. As Figure 1 shown, input the images at time T1 and T2 into the encoding-decoding networks of the upper and lower branches of the backbone feature extraction network respectively for feature extraction to obtain two feature maps F1 and F2 at different time phases;

[0033] Stack the images at time T1 and T2 in the channel dimension and then input them into the encoding-decoding network of the middle branch of the backbone feature extraction network for feature fusion and feature extraction to obtain the fused change feature map F3.

[0034] In this embodiment, the three-branch encoding-decoding network includes a 7×7 convolution with a stride of 2, a Maxpool layer, four encoding modules, and four decoding modules. The input data size of this network is W × H, and the output size is W / 4×H / 4. The encoding module is composed of two transposed convolutions, three convolutional layers, and a residual connection. The residual connection can avoid gradient disappearance and improve the feature extraction ability of the network. The decoding module is composed of two convolutional layers and one transposed convolutional layer. The encoding module uses a convolutional neural network to extract features at different scales, and the decoding network fuses the feature information at different scales through convolution and transposed convolution.

[0035] In this embodiment, the images at time T1 and T2 need to be preprocessed by matching and calibration before being input into the backbone feature network.

[0036] Step 3: Input the two feature maps F1 and F2 at different time phases and the fused change feature map F3 into a neural network model for feature enhancement, fusion, and binarization processing to obtain the change detection result, as Figure 2 shown.

[0037] In this embodiment, the neural network model used in step 3 is the Change Features Improved Network (CFI-Net). The network structure of the CFI-Net includes an unsupervised learning module for enhancing the features of the unchanged regions in two time phases and a feature fusion module for fusing the difference feature F c and the fused change feature F3.

[0038] In this embodiment, the unsupervised learning module includes a Sigmoid classifier, a threshold converter, and two multipliers. The fused change feature map F3 is input into the Sigmoid classifier to obtain a classification result, and the classification result is input into the threshold converter for threshold conversion to obtain the unchanged feature map M uc . Specifically, the threshold converter includes the implementation of binary conversion g(x) and logical NOT. The implementation of the threshold converter h(x) is specifically:

[0039] h(x) =!(g(x))

[0040] where! represents logical NOT, and the implementation of binary conversion is specifically:

[0041]

[0042] where x is the pixel value of each pixel point in the feature map.

[0043] The unchanged feature map M is calculated using the following formula uc :

[0044] M uc = h(f sig (F3))

[0045] where f sig (x) represents the Sigmoid function.

[0046] The unchanged feature map M uc is multiplied by F1 and F2 respectively to obtain the unchanged feature maps F uc1 and F uc2 at time T1 and time T2,

[0047] The loss L is calculated for the unchanged feature maps F uc1 and F uc2 at time T1 and time T2, so that unsupervised learning is performed to reduce the gap between the unchanged features F mse and F uc1 and F uc2 . In this embodiment, by reducing the difference in the unchanged regions of the feature maps F1 and F2, the difference feature F is strengthened in the reverse direction cThe differential feature F c is obtained by subtracting the feature maps F1 and F2. By reducing the gap of its unchanged features, the changing features are strengthened, further improving the accuracy of change detection, which is beneficial to reducing the human, time, and financial costs in land law enforcement work.

[0048] The loss function L mse The calculation process can be expressed as:

[0049]

[0050] where i represents the i-th pixel, and n represents the total number of pixels. represents the i-th pixel of the feature map F uc1 , represents the i-th pixel of the feature map F uc2 .

[0051] In this embodiment, the feature fusion module for fusing the differential feature F c and the fused change feature map F3 is composed of two deconvolutions with a convolutional kernel size of 3×3, a 1×1 convolution, and an Argmax function. The differential feature F c of two time phases is obtained by subtracting the feature map F1 extracted from the image at time T1 and the feature map F2 extracted from the image at time T2. The differential feature F c is stacked with the fused change feature map F3 in the channel dimension and input into the feature fusion module for feature fusion and binarization processing to obtain the final binary image. The binarization processing is performed using the Argmax function. In this embodiment, fusing the differential feature F c with the fused change feature map F3 is mainly to obtain richer change feature information. As Figure 3 shown, the technical solution of the present invention is applied to the detection and auxiliary supervision of rural illegal construction on cultivated land. The specific implementation method verifies the effectiveness of the rural illegal construction on cultivated land remote sensing image change detection method and system based on CFI-Net with this work as an example. As Figure 3 shown, where Figure 3 (a) is the image at time T1, Figure 3 (b) is the image at time T2, Figure 3 (c) is the detection result image of the method proposed by the present invention, Figure 3 (d) is the reference image. It can be seen from Figure 3 (c) and Figure 3 (d) that the difference between the detection result of the method proposed by the present invention and the reference image is small, indicating that the rural illegal construction on cultivated land remote sensing image change detection method based on CFI-Net proposed by the present invention can effectively detect the problem of rural illegal construction on cultivated land.

[0052] In this embodiment, the training samples of the Change Feature Enhancement Network (CFI-Net) are from Google satellite images of different time phases. Using the PyTorch deep learning framework, the CFI-Net is built. The labeled training samples are cut into images of size 512*512, and the data is allocated according to the ratio of training set:validation set:test set being 4:1:1. The training uses the Adam optimizer with an initial learning rate of 0.001. When the loss does not decrease for 15 consecutive times, the learning rate is reduced to half of the previous value. When the training reaches the maximum number of iterations, the model training ends.

[0053] The present invention also provides a remote sensing image change detection device, including the following modules:

[0054] Feature extraction module for different time phases: used to extract features from two remote sensing images at time T1 and time T2 of the same area, obtaining two feature maps F1 and F2 of different time phases;

[0055] Changed feature map extraction module: used to stack the images at time T1 and time T2 in the channel dimension, perform feature fusion, and extract the fused changed feature map F3;

[0056] Detection result output module: input the two feature maps F1 and F2 of different time phases and the fused changed feature F3 into the neural network model for feature enhancement and fusion to obtain the change detection result.

[0057] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A remote sensing image change detection method, characterized in that, It includes the following steps: Step 1: Extract features from two remote sensing images at time T1 and time T2 in the same area to obtain two feature maps F1 and F2 with different time phases; Step 2: Stack the images at time T1 and time T2 in the channel dimension and then perform feature fusion to extract the fused change feature map F3; Step 3: Input the two feature maps F1 and F2 with different time phases and the fused change feature map F3 into a neural network model for feature enhancement, fusion, and binarization processing to obtain the change detection result; The method for extracting the two feature maps F1 and F2 with different time phases and the fused change feature map F3 is to use a backbone feature extraction network; The backbone feature extraction network is composed of a three-branch encoder-decoder network. Input the images at time T1 and time T2 into the encoder-decoder networks of the upper and lower branches of the backbone feature extraction network respectively for feature extraction to obtain two feature maps F1 and F2 with different time phases; Stack the images at time T1 and time T2 in the channel dimension and then input them into the encoder-decoder network of the middle branch of the backbone feature extraction network for feature fusion and feature extraction to obtain the fused change feature F3; The neural network model used in Step 3 is the Change Feature Enhancement Network CFI-Net. The network structure of the Change Feature Enhancement Network CFI-Net includes an unsupervised learning module for feature enhancement of the unchanged region features in two time phases and a feature fusion module for feature fusion of the difference feature F c and the fused change feature map F3. The difference feature F c refers to the difference between the feature map F1 extracted from the image at time T1 and the feature map F2 extracted from the image at time T2 for two time phases; The unsupervised learning module includes a Sigmoid classifier, a threshold converter, and two multipliers. The fused change feature F3 is input into the Sigmoid classifier to obtain a classification result, and the classification result is input into the threshold converter for threshold conversion to obtain the unchanged change feature map M uc ; Multiply the unchanged feature map M uc with F1 and F2 respectively to obtain the unchanged feature maps F at time T1 and T2 uc1 and F uc2， For the feature maps F that remain unchanged at times T1 and T2 uc1 and F uc2 calculate the loss L mse such that the loss L mse is smaller to reduce the difference between the unchanged feature maps F uc1 and F uc2 thereby narrowing the gap between them.

2. The detection method according to claim 1, characterized in that, The differential feature F c and the feature fusion module for fusing the fused change feature map F3 consists of two transposed convolutions with a kernel size of 3×3, a 1×1 convolution, and an Argmax function. The differential feature F c and the fused change feature map F3 are stacked in the channel dimension and input into the feature fusion module for feature fusion and binarization processing to obtain the final binary image.

3. A remote sensing image change detection device, characterized in that, It includes the following modules: Feature extraction module for different time phases: Used to extract features from two remote sensing images at time T1 and time T2 in the same area to obtain two feature maps F1 and F2 with different time phases; Changed feature map extraction module: Used to stack the images at time T1 and time T2 in the channel dimension, then perform feature fusion and extract the fused change feature map F3; The method for extracting the two feature maps F1 and F2 with different time phases and the fused change feature map F3 is to use a backbone feature extraction network; The backbone feature extraction network is composed of a three-branch encoder-decoder network. Input the images at time T1 and time T2 into the encoder-decoder networks of the upper and lower branches of the backbone feature extraction network respectively for feature extraction to obtain two feature maps F1 and F2 with different time phases; Stack the images at time T1 and time T2 in the channel dimension and then input them into the encoder-decoder network of the middle branch of the backbone feature extraction network for feature fusion and feature extraction to obtain the fused change feature F3; Detection result output module: Input the feature maps F1 and F2 of two different time phases and the fused change feature map F3 into a neural network model for feature enhancement and fusion to obtain a change detection result; the neural network model used by the detection result output module is the change feature enhancement network CFI-Net. The network structure of the change feature enhancement network CFI-Net includes an unsupervised learning module for feature enhancement of the unchanged region features of two time phases and a feature fusion module for feature fusion of the difference feature F c and the fused change feature map F3. The difference feature F c refers to the difference between the feature map F1 extracted from the image at time T1 and the feature map F2 extracted from the image at time T2 for two time phases; the unsupervised learning module includes a Sigmoid classifier, a threshold converter, and two multipliers. Input the fused change feature F3 into the Sigmoid classifier to obtain a classification result, and input the classification result into the threshold converter for threshold conversion to obtain the unchanged change feature map M uc ; Multiply the unchanged change feature map M uc with F1 and F2 respectively to obtain the unchanged feature maps F uc1 and F uc2 at time T1 and time T2. Calculate the loss L uc1 for the unchanged feature maps F uc2 at time T1 and time T2, and make the loss L mse smaller to reduce the gap between the unchanged feature maps F mse and F uc1 and F uc2 .

Images