A Remote Sensing Image Change Detection Method and System

Abstract

A remote sensing image change detection method and system disclosed by the present invention include: obtaining reflection maps of a first remote sensing image and a second remote sensing image; obtaining a change detection map according to the two reflection maps and a trained change detection network; the change detection network includes a plurality of upsampling modules and a plurality of convolution modules, stacking the two reflection maps and inputting them into the change detection network, performing continuous convolution through the plurality of convolution modules, and outputting the feature maps obtained by each convolution module; the feature map output by the last convolution module is continuously upsampled through the plurality of upsampling modules, stacking the feature maps output by each upsampling module with the feature maps output by the convolution modules of the same size, performing upsampling or downsampling on each stacked feature map to obtain a feature map of the same size as the other stacked feature maps; adding the feature maps of the same size; performing convolution on the added image to obtain a change detection map. An accurate change detection map can be obtained.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image change detection, and in particular to a remote sensing image change detection method and system. Background Technique

[0002] The statements in this part only provide background technical information related to the present invention and do not necessarily constitute prior art.

[0003] Change detection technology usually performs change detection on images of the same area at different time periods. Currently, change detection technology has been applied in many fields, such as land use, urban expansion, farmland change, forest protection, etc. With the rapid development of deep learning, change detection methods based on deep learning have been continuously proposed, including convolutional neural networks, deep neural networks, etc.

[0004] The core of change detection technology is the extraction of image change information. The extraction of image change information mainly has the following three categories: 1) Pixel-based change information extraction. This type of algorithm takes pixels as processing units and calculates change information pixel by pixel. This method has a good effect on change detection of large areas, but is prone to noise. 2) Object-oriented change detection. This type of method is based on image segmentation and classification and compares features in units of objects. 3) Time series analysis method. This type of method is for long-term impact analysis, but has relatively high requirements for time resolution. 4) Deep learning method. This type of method has an end-to-end network structure and is currently widely applied to change detection technology, including convolutional neural networks, deep neural networks, recurrent neural networks, etc. When this method performs change detection on remote sensing images, it directly extracts features from the remote sensing images and directly identifies the extracted features, without considering the influence of pseudo-changes caused by weather seasons, etc. on remote sensing images on the detection results, resulting in inaccurate detection of remote sensing image changes. Summary of the Invention

[0005] In order to solve the above problems, the present invention proposes a remote sensing image change detection method and system, which reduces the influence of "pseudo-changes" on the detection results by extracting the reflectance map of the remote sensing image, and can obtain an accurate change detection map by identifying the extracted reflectance map through a change detection network.

[0006] To achieve the above object, the present invention adopts the following technical solutions:

[0007] In a first aspect, a remote sensing image change detection method is proposed, including:

[0008] Obtain a first remote sensing image and a second remote sensing image;

[0009] Obtain the reflectance maps of the first remote sensing image and the second remote sensing image;

[0010] Based on two reflection maps and a trained change detection network, a change detection map is obtained;

[0011] Among them, the change detection network includes a feature extraction module and multiple upsampling modules. The feature extraction module includes multiple convolutional modules. After stacking the two reflection maps and inputting them into the change detection network, continuous convolution is performed through multiple convolutional modules, and the feature maps obtained by each convolutional module are output; the feature map output by the last convolutional module is continuously upsampled through multiple upsampling modules. The feature maps output by each upsampling module are stacked with the feature maps output by convolutional modules of the same size and then the stacked feature maps are output. Each stacked feature map is upsampled or downsampled to obtain a feature map of the same size as the other stacked feature maps; the feature maps of the same size are added together to obtain multiple added images; the added images are convolved to obtain the change detection map.

[0012] In a second aspect, a remote sensing image change detection system is proposed, including:

[0013] A remote sensing image acquisition module for acquiring a first remote sensing image and a second remote sensing image;

[0014] A reflection map extraction module for obtaining the reflection maps of the first remote sensing image and the second remote sensing image;

[0015] A change detection map acquisition module for obtaining a change detection map based on two reflection maps and a trained change detection network;

[0016] Among them, the change detection network includes a feature extraction module and multiple upsampling modules. The feature extraction module includes multiple convolutional modules. After stacking the two reflection maps and inputting them into the change detection network, continuous convolution is performed through multiple convolutional modules, and the feature maps obtained by each convolutional module are output; the feature map output by the last convolutional module is continuously upsampled through multiple upsampling modules. The feature maps output by each upsampling module are stacked with the feature maps output by convolutional modules of the same size and then the stacked feature maps are output. Each stacked feature map is upsampled or downsampled to obtain a feature map of the same size as the other stacked feature maps; the feature maps of the same size are added together to obtain multiple added images; the added images are convolved to obtain the change detection map.

[0017] In a third aspect, an electronic device is proposed, including a memory and a processor, as well as computer instructions stored on the memory and running on the processor. When the computer instructions are run by the processor, the steps of a remote sensing image change detection method are completed.

[0018] Fourthly, a computer-readable storage medium is proposed for storing computer instructions, which, when executed by a processor, complete the steps of a remote sensing image change detection method.

[0019] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0020] 1. The present invention reduces the influence of "false changes" on the detection result by extracting the reflection map of the remote sensing image, and can obtain an accurate change detection map by identifying the extracted reflection map through a change detection network.

[0021] 2. The change detection network of the present invention performs multiple convolutions on the input image, continuously upsamples the feature map output by the last convolution, stacks the feature map obtained by each upsampling with the feature map output by the convolution of the same size, adjusts each stacked feature map to the feature map of the same size as the other stacked feature maps, and superimposes the feature maps of the same size to obtain multiple superimposed images; convolves the superimposed images to obtain a change detection map, which can obtain more complete feature information of the reflection map, thereby improving the accuracy of the change detection map.

[0022] Advantages of additional aspects of the present invention will be partially given in the following description, partially become apparent from the following description, or be understood through the practice of the present invention. Description of the Drawings

[0023] The specification drawings constituting a part of this application are used to provide a further understanding of this application. The schematic embodiments of this application and their descriptions are used to explain this application and do not constitute an improper limitation of this application.

[0024] Figure 1 It is a network architecture diagram of the method disclosed in Embodiment 1;

[0025] Figure 2 It is the first remote sensing image obtained in Embodiment 1;

[0026] Figure 3 It is the second remote sensing image obtained in Embodiment 1;

[0027] Figure 4 It is the change detection image obtained by performing change detection on the first remote sensing image and the second remote sensing image by the method disclosed in Embodiment 1, with a size of 1x256x256;

[0028] Figure 5 It is the change detection image obtained by performing change detection on the first remote sensing image and the second remote sensing image by other methods in Embodiment 1, with a size of 1x256x256;

[0029] Figure 6The ground truth of change detection for the first remote sensing image and the second remote sensing image obtained in Example 1. Detailed implementation manners

[0030] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0031] It should be noted that the following detailed descriptions are all illustrative and are intended to provide further descriptions of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present application belongs.

[0032] Example 1

[0033] In this embodiment, a method for change detection of remote sensing images is disclosed. As Figure 1 shown, Figure 1 where c represents a stacking operation, including:

[0034] S1: Obtain a first remote sensing image and a second remote sensing image.

[0035] In specific implementation, a first remote sensing image T1 and a second remote sensing image T2 are obtained, and the sizes of the first remote sensing image T1 and the second remote sensing image T2 are 3×256×256.

[0036] S2: Obtain the reflection maps of the first remote sensing image and the second remote sensing image. The process is as follows:

[0037] S21: Extract remote sensing image features from the first remote sensing image and the second remote sensing image respectively.

[0038] Use a feature extraction module to extract remote sensing image features from the first remote sensing image T1 and the second remote sensing image T2 respectively.

[0039] The feature extraction module uses a convolution operation to extract remote sensing image features from the remote sensing image. Specifically:

[0040] F = Φ(σ(Φ(CBAM(σ(Φ(σ(Φ(T i ))))))))

[0041] where F is the extracted remote sensing image feature, with a size of 32×256×256; σ represents LeakyRelu, Φ represents a convolution operation, the convolution kernel size is 3×3, CBAM represents a convolutional attention module, and T i is the obtained remote sensing image, and i = 1 or 2.

[0042] S22: Decompose the two remote sensing image features respectively to obtain their respective reflection maps and illumination maps.

[0043] In this embodiment, the remote sensing features obtained from the first remote sensing image and the remote sensing features obtained from the second remote sensing image are respectively subjected to intrinsic image decomposition (Retinex decomposition) to obtain the reflection map R1 and illumination map I1 of the first remote sensing image, and the reflection map R2 and illumination map I2 of the second remote sensing image.

[0044] In specific implementation, the remote sensing feature F is respectively input into I-Net and R-Net for Retinex decomposition to obtain the reflection map R and illumination map I of the original remote sensing image.

[0045] Among them, I-Net includes 3 modules composed of a 3×3 convolutional layer and a layer of LeakyRelu, and a module composed of 2 layers of 3×3 convolutional layers, a layer of LeakyRelu, a layer of 1×1 convolutional layer, and a layer of Sigmod function layer. After the remote sensing feature F obtained by S21 passes through the two modules in I-Net successively, the illumination map I of the original remote sensing image can be obtained, and the size of the illumination map I is 1×256×256.

[0046] R-Net includes 3 modules composed of a 3×3 convolutional layer, a layer of LeakyRelu, and a residual module, and a module composed of two 3×3 convolutional layers, a layer of LeakyRelu, and a layer of Sigmod function layer. After the remote sensing feature F obtained by S21 passes through the two modules in R-Net successively, the reflection map R of the original remote sensing image can be obtained, and the size of the reflection map R is 3×256×256.

[0047] That is, the remote sensing feature F is input into I-Net and successively passes through 3 layers of 3x3 convolution, LeakyRelu, 2 layers of 3x3 convolution, LeakyRelu, a layer of 1x1 convolution, and a layer of Sigmod layer to output the illumination map I.

[0048] The remote sensing feature F is input into R-NET and successively passes through 3 layers of 3x3 convolution, LeakyRelu, a layer of residual module, 2 layers of 3x3 convolution, LeakyRelu, and a layer of Sigmod layer to output the reflection map R.

[0049] S3: Obtain a change detection map according to the two reflection maps and the trained change detection network;

[0050] Among them, the change detection network includes a feature extraction module and multiple upsampling modules. The feature extraction module includes multiple convolutional modules. After stacking two reflection maps and inputting them into the change detection network, continuous convolution is performed through multiple convolutional modules, and the feature maps obtained by each convolutional module are output. The feature map output by the last convolutional module is continuously upsampled through multiple upsampling modules. The feature map output by each upsampling module is stacked with the feature map output by the convolutional module of the same size and then the stacked feature map is output. Each stacked feature map is upsampled or downsampled to obtain a feature map of the same size as the other stacked feature maps. The feature maps of the same size are superimposed to obtain multiple superimposed images. Convolution is performed on the superimposed images to obtain the change detection map.

[0051] In this embodiment, after stacking the reflection map R1 and the reflection map R2, they are input into the trained change detection network to obtain the change detection map.

[0052] Among them, the feature extraction module of the change detection network includes multiple convolutional modules. The multiple convolutional modules have the same structure and are connected in sequence. Each convolutional module performs convolution and pooling operations on the input image, extracts features, increases channels, and downsamples the input image through convolution and pooling operations, and then outputs the feature map.

[0053] Preferably, it includes four convolutional modules, and the convolution in each convolutional module is 3x3 convolution.

[0054] The specific operation of each convolutional module is as follows:

[0055] F i = Φ(σ(δ(Φ(F i-1 )))) (i = 2, 3, 4)

[0056] Among them, F i-1 is the image input into the convolutional module; F i is the feature map output by the convolutional module.

[0057] After stacking the reflection map R1 and the reflection map R2 and inputting them into the first convolutional module, after performing 3x3 convolution operation and pooling operation, the feature map F1 is obtained; the feature map F1 is input into the second convolutional module to obtain the feature map F2; the feature map F2 is input into the third convolutional module to obtain the feature map F3; the feature map F3 is input into the fourth convolutional module to obtain the feature map F4.

[0058] The sizes of the feature maps F1, F2, F3, and F4 are 32×256×256, 32×128×128, 32×64×64, and 32×32×32 respectively.

[0059] The first upsampling module takes only the feature map output by the last convolutional module as input; except for the last upsampling module, the feature maps output by the remaining upsampling modules are stacked with the feature maps output by convolutional modules of the same size, and the stacked feature maps are output. The stacked feature maps also serve as the input to the next upsampling module.

[0060] Stack the feature map output by the last upsampling module with the feature map output by the first convolutional module to obtain the third stacked feature map. Sample the remaining stacked feature maps to the same size as the third stacked feature map and superimpose them on the third stacked feature map to obtain the superimposed image; perform convolution on the superimposed image to obtain the change detection map.

[0061] Preferably, the change detection network includes three upsampling modules, and the three upsampling modules have the same structure, each including an upsampling layer, a 3×3 convolution, a regularization layer, and a Relu layer.

[0062] In a specific implementation, input the feature map F4 output by the fourth convolutional module into the first upsampling module. The output feature map has the same size as the feature map F3 output by the third convolutional module. Stack the feature map with the feature map F3 output by the third convolutional module to obtain the stacked feature map F5. Input the feature map F5 into the second upsampling module. The feature map output by the second upsampling module has the same size as the feature map F2 output by the second convolutional module. Stack the two to obtain the stacked feature map F6; input the feature map F6 into the third upsampling module. The feature map output by the third upsampling module has the same size as the feature map F1 output by the first convolutional module. Stack the two to obtain the stacked feature map F7.

[0063] The sizes of the stacked feature maps F5, F6, and F7 are 64×64×64, 64×128×128, and 64×256×256 respectively.

[0064] Downsample the stacked feature maps F6 and F7 to the same size as the stacked feature map F5; add the two downsampled feature maps to the stacked feature map F5 to obtain the added feature map M1. The size of M1 is 219×64×64. Then input M1 into the convolutional module to obtain the first change detection map with a size of 1 / 4 of the original remote sensing image for deep supervision.

[0065] Upsample the stacked feature map F5 and downsample the stacked feature map F7 to the same size as the stacked feature map F6. Add the upsampled image of F5, the downsampled image of F7, and the stacked feature map F6 to obtain the added feature map M2. The size of M2 is 219×128×128. Then input M2 into the convolutional module to obtain the second change detection map with a size of 1 / 2 of the original remote sensing image for deep supervision.

[0066] Upsample the stacked feature maps F5 and F6 to the same size as the stacked feature map F7; add the upsampled feature maps of F5 and F6 to the stacked feature map F7 to obtain the added feature map M3, with the size of M3 being 219×256×256, and then input M3 into the convolutional module to obtain the third change detection map with the same size as the original remote sensing image, and the third change detection map is the output result of the change detection network.

[0067] The convolutional module input by the stacked feature maps contains two layers of 3×3 convolution and one layer of Sigmod.

[0068] In this embodiment, after constructing the change detection network according to the above structure, obtain the training data, and use the training data to train the change detection network by using the stochastic gradient descent algorithm. After training is completed, obtain the trained change detection network.

[0069] Among them, the training data uses the data in the public large building change detection dataset, which includes 637 pairs of high-resolution (0.5m) remote sensing images with the size of 1024×1024. Cut the images into small pieces with the size of 3×256×256, and there is no overlap between the small pieces to obtain paired training data.

[0070] In specific implementation, the loss function L of Retinex decomposition is:

[0071] L = L1 + L2 + L C

[0072] L1 = ||T1 - I1⊙R1|| 2 + αE s (I1) + βE t (R1, I1)

[0073] L2 = ||T2 - I2⊙R2|| 2 + αE s (I2) + βE t (R2, I2)

[0074]

[0075] Among them, T1 and T2 are the first remote sensing image and the second remote sensing image respectively, R1 and R2 are the reflection maps of the first remote sensing image and the second remote sensing image respectively, I1 and I2 are the illumination maps of the first remote sensing image and the second remote sensing image respectively, M is the reference of the final change detection result, α and β are the parameters controlling the proportion of the sub-loss functions E S and E T L1 is the loss function applied to the first remote sensing image, L2 is the loss function applied to the second remote sensing image, LC is a loss function applied to the reflection maps of the first remote sensing image and the second remote sensing image.

[0076] E S (I) = ||I - I0|| 1

[0077]

[0078]

[0079]

[0080] where T is the original remote sensing image, I is the illumination map decomposed from the remote sensing image, and I0 is the result after taking the maximum value of each pixel point of the original image T in the channel dimension. refers to taking the maximum value of each pixel point of the image in the channel dimension, and W is the weight matrix. is the first derivative operator. refers to taking derivatives in the vertical and horizontal directions respectively, and exp is the exponential function.

[0081] The loss function L of the change detection network CD is:

[0082] L CD = L DICE + L BCE

[0083]

[0084]

[0085] where a is a parameter used to solve the problem of imbalance in the number of changed pixels and unchanged pixels, y i is the value of the corresponding pixel in the change result reference map, and p i is the value of the corresponding pixel point in the change detection map.

[0086] The data in the public large building change detection dataset is adopted, including 637 pairs of high-resolution (0.5m) remote sensing images, with the image size of 1024×1024 and the time span of 5 to 14 years of bit images. The effects of the method disclosed in this embodiment and the fully convolutional early fusion network (FC-EF) are compared and verified, and the results are as Figures 2 - 6As shown, it can be seen that the detection result of the method disclosed in this embodiment is basically consistent with the true value, achieving a very accurate change detection effect. Conducting an objective index evaluation on the detection result, as shown in Table 1, the accuracy rate (P) of the detection result of the method disclosed in this embodiment reaches 0.8969, the value of the average intersection over union (IoU) reaches 0.8361, the harmonic mean (F1) of the precision rate and the recall rate reaches 0.9107, and the recall rate (R) value reaches 0.9250. It can be seen from this that this embodiment has achieved a relatively high accuracy rate and precision rate in change detection and can obtain accurate change detection results.

[0087] Table 1 Objective Index Evaluation Results

[0088] P(％) R(％) F1(％) IoU(%) FC-EF 71.63 67.25 69.37 53.11 The method disclosed in this embodiment 89.69 92.5 91.07 83.61

[0089] The method disclosed in this embodiment reduces the influence of "false changes" on the detection result by extracting the reflectance map of the remote sensing image, and can obtain an accurate change detection map by identifying the extracted reflectance map through a change detection network.

[0090] Embodiment 2

[0091] In this embodiment, a remote sensing image change detection system is disclosed, including:

[0092] A remote sensing image acquisition module for acquiring a first remote sensing image and a second remote sensing image;

[0093] A reflectance map extraction module for acquiring the reflectance maps of the first remote sensing image and the second remote sensing image;

[0094] A change detection map acquisition module for obtaining a change detection map according to the two reflectance maps and a trained change detection network;

[0095] Among them, the change detection network includes a feature extraction module and multiple upsampling modules. The feature extraction module includes multiple convolutional modules. After stacking the two reflectance maps and inputting them into the change detection network, continuous convolution is performed through multiple convolutional modules, and the feature maps obtained by each convolutional module are output; the feature map output by the last convolutional module is continuously upsampled through multiple upsampling modules, and the feature map output by each upsampling module is stacked with the feature map output by the convolutional module of the same size and then the stacked feature map is output. Each stacked feature map is upsampled or downsampled to obtain a feature map of the same size as the other stacked feature maps; the feature maps of the same size are added together to obtain multiple added images; convolution is performed on the added images to obtain a change detection map.

[0096] Embodiment 3

[0097] In this embodiment, an electronic device is disclosed, which includes a memory, a processor, and computer instructions stored on the memory and running on the processor. When the computer instructions are run by the processor, the steps of a remote sensing image change detection method disclosed in Embodiment 1 are completed.

[0098] Embodiment 4

[0099] In this embodiment, a computer-readable storage medium is disclosed, which is used to store computer instructions. When the computer instructions are executed by the processor, the steps of a remote sensing image change detection method disclosed in Embodiment 1 are completed.

[0100] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the above embodiments, those of ordinary skill in the art should understand that: the specific implementation manners of the present invention can still be modified or equivalently replaced, and any modification or equivalent replacement without departing from the spirit and scope of the present invention shall be covered by the protection scope of the claims of the present invention.

Claims

1. A method for remote sensing image change detection, characterized in that, Including: Obtain a first remote sensing image and a second remote sensing image; Obtain the reflection maps of the first remote sensing image and the second remote sensing image; Obtain a change detection map according to the two reflection maps and a trained change detection network; Among them, the change detection network includes a feature extraction module and multiple upsampling modules. The feature extraction module includes multiple convolutional modules. The two reflection maps are stacked and input into the change detection network, and continuous convolution is performed by multiple convolutional modules, and the feature maps obtained by each convolutional module are output; the feature map output by the last convolutional module is continuously upsampled by multiple upsampling modules, and the feature map output by each upsampling module is stacked with the feature map output by the convolutional module of the same size and then the stacked feature map is output. Each stacked feature map is upsampled or downsampled to obtain a feature map of the same size as the other stacked feature maps; the feature maps of the same size are added to obtain multiple added images; the added images are convolved to obtain a change detection map; Each convolutional module performs convolution and pooling operations on the input image, and then outputs a feature map; The first upsampling module takes only the feature map output by the last convolutional module as input; except for the last upsampling module, the feature map output by the other upsampling modules is stacked with the feature map output by the convolutional module of the same size, and the stacked feature map is output. The stacked feature map is also used as the input of the next upsampling module; The feature map output by the last upsampling module is stacked with the feature map output by the first convolutional module to obtain a third stacked feature map. The other stacked feature maps are sampled to the same size as the third stacked feature map and added to the third stacked feature map to obtain an added image; the added image is convolved to obtain a change detection map.

2. The remote sensing image change detection method according to claim 1, wherein, Extract remote sensing image features from the first remote sensing image and the second remote sensing image respectively; Decompose the two remote sensing image features respectively to obtain their respective reflection maps and illumination maps.

3. The remote sensing image change detection method according to claim 1, characterized in that, Use the stochastic gradient descent algorithm to train the change detection network.

4. A remote sensing image change detection system, characterized in that, Including: A remote sensing image acquisition module for acquiring a first remote sensing image and a second remote sensing image; A reflection map extraction module for acquiring the reflection maps of the first remote sensing image and the second remote sensing image; A change detection map acquisition module for obtaining a change detection map according to the two reflection maps and a trained change detection network; Among them, the change detection network includes a feature extraction module and multiple upsampling modules. The feature extraction module includes multiple convolutional modules. The two reflection maps are stacked and input into the change detection network, and continuous convolution is performed by multiple convolutional modules, and the feature maps obtained by each convolutional module are output; the feature map output by the last convolutional module is continuously upsampled by multiple upsampling modules, and the feature map output by each upsampling module is stacked with the feature map output by the convolutional module of the same size and then the stacked feature map is output. Each stacked feature map is upsampled or downsampled to obtain a feature map of the same size as the other stacked feature maps; the feature maps of the same size are added to obtain multiple added images; the added images are convolved to obtain a change detection map; Each convolutional module performs convolution and pooling operations on the input image, and then outputs a feature map. The first upsampling module takes only the feature map output by the last convolutional module as input; except for the last upsampling module, the feature maps output by the remaining upsampling modules are stacked with the feature maps output by convolutional modules of the same size, and the stacked feature maps are output, which also serve as the input for the next upsampling module. The feature map output by the last upsampling module is stacked with the feature map output by the first convolutional module to obtain the third stacked feature map. The remaining stacked feature maps are sampled to the same size as the third stacked feature map and added to the third stacked feature map to obtain the added image. Convolution is performed on the added image to obtain the change detection map.

5. An electronic device, characterized in that, It includes a memory, a processor, and computer instructions stored on the memory and running on the processor. When the computer instructions are run by the processor, the steps of a remote sensing image change detection method according to any one of claims 1-3 are completed.

6. A computer-readable storage medium, characterized in that, For storing computer instructions, when the computer instructions are executed by the processor, the steps of a remote sensing image change detection method according to any one of claims 1-3 are completed.

Images