Optical remote sensing image building change detection method based on deep learning

Through deep learning technology, combined with the UHRNet model and attention module, the problems of low detection accuracy and incomplete edges of optical remote sensing images are solved, achieving more accurate change detection and better universality.

CN119007001AActive Publication Date: 2024-11-22UNIV OF ELECTRONICS SCI & TECH OF CHINA

Patent Information

Application Number
CN202410943718.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-15
Publication Date
2024-11-22
Estimated Expiration
2044-07-15

AI Technical Summary

Technical Problem

The existing building change detection methods have low detection accuracy of optical remote sensing images, incomplete detection edges, and poor universality of change detection algorithms on different data sets.

Method used

Using the optical remote sensing image building change detection method based on deep learning, the UHRNet model combines the encoding-decoder network structure, introduces channel and spatial attention modules, and uses edge guidance modules to perform feature extraction and enhancement processing of differential feature maps to achieve More accurate change detection.

Benefits of technology

Improves the accuracy of changing feature modeling and extraction, enhances the integrity of detection edges, and shows high universality and detection accuracy on different data sets.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119007001A_ABST
    Figure CN119007001A_ABST
Patent Text Reader

Abstract

The invention discloses an optical remote sensing image building change detection method based on deep learning, and belongs to the field of remote sensing image change detection. The method comprises the following steps: firstly, acquiring optical remote sensing image data from high-resolution No.2, preprocessing the optical remote sensing image data, and labeling a change information label Label; and then the image is input into the change detection model, and a detection result graph is output through calculation. The problems that an existing change detection method is low in detection precision and incomplete in detection edge, and a change detection algorithm is poor in universality on different data sets are effectively optimized and improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of building change detection in optical remote sensing images. Specifically, it uses deep learning technology to extract building change information, thereby providing services for specific businesses such as land monitoring. Background Art

[0002] With the acceleration of urbanization, buildings, as an important part of urban space, have extremely important change detection significance in urban planning, land use management, disaster monitoring and other aspects. Traditional building change detection methods often rely on manual visual interpretation, which has problems such as low efficiency and strong subjectivity. In recent years, satellite remote sensing technology has developed rapidly, especially the optical remote sensing satellite "Gaofen-2" launched by my country, which has achieved sub-meter spatial resolution. However, the increase in resolution also means a sharp increase in data volume, which brings opportunities and challenges to the field of change detection. At the same time, computer vision technology has also been continuously improving, and the building change detection method based on deep learning in optical remote sensing images has gradually become a research hotspot.

[0003] There are two types of change detection methods for remote sensing images: remote sensing change detection based on traditional methods and remote sensing change detection based on deep learning. Traditional methods rely too much on expert experience and preprocessing operations, which results in a cumbersome detection process, low detection accuracy, more manual intervention, low degree of automation, and high time consumption. Remote sensing change detection based on deep learning can automatically extract the deep features of remote sensing image data, abstract features that are more suitable for the current task, and obtain more accurate results. It can also achieve end-to-end learning without manual intervention. And as the amount of data increases, the performance of the model will usually be improved. These characteristics are in line with the trend of remote sensing images developing towards high resolution.

[0004] Research on change detection based on deep learning can be divided into two directions. One is to fuse the two or more remote sensing images for change detection at the input stage, and then send them into the convolutional neural network to finally obtain a binary change map. Another detection method is to use the twin network structure to extract the features of the two input images respectively, and then compare the extracted features to obtain the change detection result map.

[0005] At present, there are many methods for detecting building changes in remote sensing images based on deep learning at home and abroad. There is the LSTM model based on recurrent neural networks, which can process remote sensing image sequences with time order very well, but its image feature extraction ability is relatively poor; there is also the BIT model based on the Transformer mechanism, which can obtain the global attention of the image and capture long-term dependencies very well, but it requires a lot of computing resources and a large-scale data set; there is the U-Net model based on convolutional neural networks, which can extract the feature information of the image very well and has low requirements for the amount of data, but its global attention is insufficient; currently, research on the U-Net model based on convolutional neural networks is very popular, and a large number of researchers have improved the performance of the network by adding jump connections and designing attention modules. Summary of the invention

[0006] The purpose of the present invention is to provide a method for detecting building changes in optical remote sensing images based on deep learning, and to describe the entire process from obtaining raw data from optical remote sensing satellites to finally obtaining a change detection prediction map after being processed by a designed model. The present invention improves the accuracy of modeling and extracting change features, and solves the problems of low detection accuracy of optical remote sensing images, incomplete detection edges, and poor universality of change detection algorithms on different data sets in existing building change detection methods.

[0007] The technical solution adopted by the present invention is as follows:

[0008] A method for building change detection in optical remote sensing images based on deep learning

[0009] Step 1: Obtain multispectral images and panchromatic images of the target study area at two different periods, and use ENVI software to perform radiometric calibration, atmospheric correction, and orthorectification on the multispectral images, and radiometric calibration and orthorectification on the panchromatic images. Then, align the processed multispectral images and panchromatic images and perform image fusion to obtain the final remote sensing image;

[0010] Step 2: Use LabCD software to manually mark the changed parts of the buildings in the two remote sensing images of different phases, obtain the corresponding change Label map, and then use the Sobel operator to obtain the edge label of the corresponding change area;

[0011] Step 3: preprocessing the acquired two-phase optical remote sensing image data;

[0012] Step 4: Input the read image T1 before the change and the image T2 after the change into the weight-sharing UHRNet encoder respectively, and extract the feature maps of the respective images at different levels from stage 1 to stage 5;

[0013] Step 5: The feature maps of the same stage of T1 and T2 are respectively sent to the difference feature extraction module to obtain the difference feature maps of the remote sensing feature images of each stage. The calculation formula is as follows:

[0014] D i =abs(X i1 ,X i2 )

[0015] Where X i1 ,X i2 Denote the feature maps of T1 and T2 of the i-th stage, D i represents the output of the i-th stage;

[0016] Step 6: Send the difference feature maps of each stage to the channel and spatial attention modules respectively. The channel attention module can calculate the importance of each channel and give different attention to each channel; the spatial attention module can assign different weights to different positions of the input data, so that the model pays more attention to important information, thereby obtaining an enhanced difference feature map;

[0017] Step 7: Feed the minimum resolution difference feature map into the UHRNet decoder and add a skip connection in the decoder;

[0018] Step 8: The output of the UHRNet decoder is passed through the edge guidance module, and then concatenated with itself and sent to the classifier to obtain the final change detection binary image, and deep supervision is added to accelerate model convergence.

[0019] Furthermore, the specific steps of step 1 are as follows:

[0020] Step 1.1: Perform radiometric calibration on multispectral images and panchromatic images first, which can convert the digital quantization value DN into a radiometric brightness value with physical meaning, thereby more accurately describing the reflection or emission characteristics of the object. The formula is as follows:

[0021] Rad=gain*DN+Bias

[0022] Where grain is the gain coefficient, DN is the digital quantization value in the image, and Bias is the offset;

[0023] Step 1.2: Use the FLAASH atmospheric correction tool to perform atmospheric correction on multispectral images;

[0024] Step 1.3: Use RPC information to orthorectify the multispectral image and panchromatic image, and choose cubic convolution as the resampling method;

[0025] Step 1.4: Use the NNDiffuse Pan Sharpening method in ENVI to fuse the multispectral image and the panchromatic image to obtain a remote sensing image with both high spatial resolution and spectral resolution.

[0026] Furthermore, the specific steps of step 3 are as follows:

[0027] Step 3.1: After obtaining the multi-temporal optical remote sensing image data, segment it into non-overlapping image blocks with a resolution of 256*256, and divide it into three data sets: training set, validation set, and test set according to the data ratio of 7:1:2;

[0028] Step 3.2: Read the segmented remote sensing image;

[0029] Step 3.3: Perform data augmentation operations on the read training set, including random flipping in the horizontal direction, random flipping in the vertical direction, random cropping of the image, and Gaussian blurring of the image, to increase the data complexity during the training process and enhance the robustness of the model.

[0030] Furthermore, the specific steps of step 4 are as follows:

[0031] Step 4.1: For the input feature map X, perform two consecutive two-dimensional convolutions with a step size of 2 to obtain L feature maps A with a resolution of 1 / 4 of the input. The calculation formula is as follows:

[0032] A=ReLU(BN(Conv(ReLU(BN(Conv(x))))))

[0033] Step 4.2: Each stage of the UHRNet encoder consists of only two parallel HR modules, and convolutional features are extracted and two-scale resolution features are fused in each stage. Finally, each stage outputs two feature maps with a resolution difference of one times.

[0034] Step 4.3: Between every two adjacent stages of the UHRNet encoder, the output feature map with a smaller resolution in the previous stage is downsampled to obtain a feature map with a resolution reduced by half but the number of channels doubled. The feature map is used together with the resolution input before downsampling as the input of the two HR modules in the next stage, and finally 5 feature map outputs with different resolution sizes are obtained.

[0035] Furthermore, the specific steps of step 6 are as follows:

[0036] Step 6.1: D iIt is sent to the channel attention module CAM, where the maximum pooling is applied first, and then the attention weight is calculated through two fully connected layers and a sigmoid activation function. The reconstructed attention weight vector is multiplied element-by-element with the input feature map to obtain the output of the CAM. The calculation formula is as follows:

[0037] C i =σ(C2(LRe(C1(max(D i )))))

[0038]

[0039] Where C1 and C2 represent fully connected layers, LRe represents the Leaky-ReLU activation function, σ represents the sigmoid activation function, and C i represents the channel attention weight of the i-th stage, D i represents the difference feature map of the i-th stage, D ic Added the difference feature map of channel attention for the i-th layer;

[0040] Step 6.2: At the same time, the difference feature map D of each stage i It is also sent to the spatial attention module SAM, which condenses the spatial information of the feature map through max pooling and average pooling, and then calculates the spatial attention weight using the convolution layer and sigmoid activation function. The output of SAM is obtained by multiplying the obtained spatial attention weight with the input feature map element by element. The calculation formula is as follows:

[0041] S i =σ(Conv([max(D i )+avg(D i )]))

[0042]

[0043] Where S i represents the channel attention weight of the i-th stage, D i represents the difference feature map of the i-th stage, D is Added the difference feature map of spatial attention for the i-th layer;

[0044] Step 6.3: Finally, add the outputs of the channel and spatial attention modules to get the feature output D i ', the calculation formula is as follows:

[0045]

[0046] Furthermore, the specific steps of step 7 are as follows:

[0047] Step 7.1: Upsample the minimum resolution difference feature map D5' and concatenate it with the difference feature map D4' of the encoder stage 4 to obtain D4", and use D4" and D5' as input to the UHRNet decoder;

[0048] Step 7.2: For each stage of the UHRNet decoder, it consists of only two parallel HRNet branches, and convolution and fusion of two-scale resolution features are performed in each stage. Finally, each stage will output two feature maps with a resolution difference of one times.

[0049] Step 7.3: Between every two adjacent stages of the UHRNet decoder, upsample the output feature map with a larger resolution in the previous stage to obtain a feature map with half the resolution but half the number of channels, and jump-connect it to the feature map of the same resolution size of the encoder, and then use it together with the resolution input before upsampling as the input of the two HR modules of the next stage, and finally obtain 5 feature map outputs of different resolution sizes.

[0050] Furthermore, the specific steps of step 8 are as follows:

[0051] Step 8.1: For the output maximum resolution feature map D1", pass it through a 1*1 convolution module to obtain the edge prediction map, and then pass it through the softmax function to obtain the output D 1edge , then upsample to the original image size and compare with the edge label to calculate the edge loss. The corresponding loss function is:

[0052]

[0053] Step 8.2: Concatenate the output D1'' and D1'' in the channel dimension, and then send them to another 1*1 convolution module, and get the output D after the softmax function. 1cd Then upsample to the original image size and compare with the change detection label to calculate the change detection loss. The corresponding loss function is:

[0054] Loss cd =-α t (1-p t ) γ log(p t )

[0055] Step 8.3: Using deep supervision, upsample the other resolution outputs directly to the original image size and compare them with the change detection labels to calculate the change detection loss Loss ds The corresponding loss function calculation formula is the same as above; the final loss function calculation formula is as follows:

[0056] Loss=Lossedge +Loss cd +Loss ds

[0057] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0058] 1. A method for detecting building changes in optical remote sensing images based on deep learning, using an encoder-decoder network structure, combining the HRNet network structure with the U-Net network to build a U-HRNet model. The encoder part of the U-HRNet model is used to extract feature information, and the decoder part of the U-HRNet model is used to reconstruct and restore the change feature information.

[0059] 2. In step 4 of the present invention, multi-scale parallelism can be maintained at any stage of the encoder, and multi-scale fusion can be performed to well integrate the detail information contained in the low-level and the semantic information contained in the high-level, so that the information contained in the obtained feature map is richer. At the same time, compared with the HRNet network, the calculation of high-resolution feature maps is reduced, computing resources are saved, and the running time of the network is reduced.

[0060] 3. The introduction of channel and spatial attention modules in step 6 of the present invention can make the model pay more attention to important information and enhance the feature information of the difference feature map.

[0061] 4. In step 7.4 of the present invention, an edge guidance module is introduced. By using the edge labels obtained by change label processing to guide the model to learn edge features, the model can obtain a change detection prediction map with more complete edge information.

[0062] 5. The present invention is trained and tested on the Chengdu Hi-Tech Zone change detection dataset and the public dataset LEVIR-CD dataset respectively. The experimental results show that the proposed method effectively optimizes and improves the existing building change detection methods, such as low detection accuracy and incomplete detection edges on optical remote sensing images with higher resolution, and poor universality of change detection algorithms on different datasets. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] The present invention will now be described by way of example with reference to the accompanying drawings, in which:

[0064] Figure 1 It is a flowchart of a method for detecting building changes in optical remote sensing images based on deep learning;

[0065] Figure 2 This is a diagram of the overall network architecture used in Example 1 of the present invention;

[0066] Figure 3is the encoder structure adopted in Embodiment 1 of the present invention;

[0067] Figure 4 is the decoder structure adopted in Embodiment 1 of the present invention;

[0068] Figure 5 is a structural diagram of the channel and spatial attention module used in Example 1 of the present invention;

[0069] Figure 6 is a structural diagram of an edge guide module used in Example 1 of the present invention;

[0070] Figure 7 1 is a display of the change detection results of Example 1 of the present invention on the LEVIR-CD test set, wherein (a) is the image before the change, (b) is the image after the change, (c) is the change reference image, and (d) is the obtained detection result diagram;

[0071] Figure 8 This is a display of the change detection results of Example 1 of the present invention on the test set of the self-made Chengdu Hi-Tech Zone target study area change detection data set, where (a) is the image before the change, (b) is the image after the change, (c) is the change reference image, and (d) is the obtained detection result diagram. DETAILED DESCRIPTION

[0072] All features disclosed in this specification, or steps in all methods or processes disclosed, except mutually exclusive features and / or steps, can be combined in any manner.

[0073] Combine the following Figures 1 to 8 The present invention is described in detail.

[0074] Embodiment 1:

[0075] Step 1: Obtain multispectral images and panchromatic images of the target study area of ​​Chengdu Hi-tech Zone from GF-2 at two different periods, and use ENVI software to perform radiometric calibration, atmospheric correction, and orthorectification on the multispectral images, and radiometric calibration and orthorectification on the panchromatic images. Then, align the processed multispectral images and panchromatic images and perform image fusion to obtain the final remote sensing image;

[0076] Step 1.1: Perform radiometric calibration on multispectral images and panchromatic images first, which can convert digital quantization values ​​(DN values) into radiometric brightness values ​​with physical meaning, thereby more accurately describing the reflection or emission characteristics of the objects. The formula is as follows:

[0077] Rad=gain*DN+Bias

[0078] Where grain is the gain coefficient, DN is the digital quantization value in the image, and Bias is the offset;

[0079] Step 1.2: Use the FLAASH atmospheric correction tool to perform atmospheric correction on multispectral images;

[0080] Step 1.3: Use RPC information to orthorectify the multispectral image and panchromatic image, and choose cubic convolution as the resampling method;

[0081] Step 1.4: Use the NNDiffuse Pan Sharpening method in ENVI to fuse the multispectral image and the panchromatic image to obtain a remote sensing image with both high spatial resolution and spectral resolution.

[0082] Step 2: Use LabCD software to manually mark the changed parts of the buildings in the two remote sensing images of different phases, obtain the corresponding change Label map, and then use the Sobel operator to obtain the edge label of the corresponding change area;

[0083] Step 3: preprocessing the acquired two-phase optical remote sensing image data;

[0084] Step 3.1: After obtaining the multi-temporal optical remote sensing image data, segment it into non-overlapping image blocks with a resolution of 256*256, and divide it into three data sets: training set, validation set, and test set according to the data ratio of 7:1:2;

[0085] Step 3.2: Read the segmented remote sensing image;

[0086] Step 3.3: Perform data augmentation operations on the read training set, including random flipping in the horizontal direction, random flipping in the vertical direction, random cropping of the image, and Gaussian blurring of the image, to increase the data complexity during the training process and enhance the robustness of the model.

[0087] Step 4: Input the read image T1 before the change and the image T2 after the change into the weight-sharing UHRNet encoder respectively. The structure of a single UHRNet encoder is as follows: Figure 3 As shown, the feature maps of different levels of the input image from stage1 to stage5 can be extracted;

[0088] Step 4.1: For the input feature map X, perform two consecutive two-dimensional convolutions with a step size of 2 to obtain L feature maps A with a resolution of 1 / 4 of the input. The calculation formula is as follows:

[0089] A=ReLU(BN(Conv(ReLU(BN(Conv(x))))))

[0090] Step 4.2: Each stage of the UHRNet encoder consists of only two parallel HR modules, and convolutional features are extracted and two-scale resolution features are fused in each stage. Finally, each stage outputs two feature maps with a resolution difference of one times.

[0091] Step 4.3: Between every two adjacent stages of the UHRNet encoder, the output feature map with a smaller resolution in the previous stage is downsampled to obtain a feature map with a resolution reduced by half but the number of channels doubled. The feature map is used together with the resolution input before downsampling as the input of the two HR modules in the next stage, and finally 5 feature map outputs with different resolution sizes are obtained.

[0092] Step 5: Send the feature maps of the same stage of T1 and T2 to the difference feature extraction module to obtain the difference feature maps of the remote sensing feature images at each stage, such as Figure 2 As shown, X i1 ,X i2 Denote the feature maps of T1 and T2 of the i-th stage, D i represents the output of the i-th stage, and the calculation formula is as follows:

[0093] D i =abs(X i1 ,X i2 )

[0094] Step 6: Send the difference feature maps of each stage to the channel and spatial attention modules respectively. The channel and spatial attention modules are as follows: Figure 5 The channel attention module can calculate the importance of each channel and give different attention to each channel; the spatial attention module can assign different weights to different positions of the input data, so that the model pays more attention to important information, thereby obtaining an enhanced difference feature map;

[0095] Step 6.1: D i It is sent to the channel attention module CAM, where the maximum pooling is applied first, and then the attention weight is calculated through two fully connected layers and a sigmoid activation function. The reconstructed attention weight vector is multiplied element-by-element with the input feature map to obtain the output of the CAM. The calculation formula is as follows:

[0096] C i =σ(C2(LRe(C1(max(D i )))))

[0097]

[0098] Where C1 and C2 represent fully connected layers, LRe represents the Leaky-ReLU activation function, σ represents the sigmoid activation function, and C i represents the channel attention weight of the i-th stage, D i represents the difference feature map of the i-th stage, D ic Added the difference feature map of channel attention for the i-th layer;

[0099] Step 6.2: At the same time, the difference feature map D of each stage i It is also sent to the spatial attention module SAM, which condenses the spatial information of the feature map through max pooling and average pooling, and then calculates the spatial attention weight using the convolution layer and sigmoid activation function. The output of SAM is obtained by multiplying the obtained spatial attention weight with the input feature map element by element. The calculation formula is as follows:

[0100] S i =σ(Conv([max(D i )+avg(D i )]))

[0101]

[0102] Where S i represents the channel attention weight of the i-th stage, D i represents the difference feature map of the i-th stage, D is Added the difference feature map of spatial attention for the i-th layer;

[0103] Step 6.3: Finally, add the outputs of the channel and spatial attention modules to get the feature output D i ', the calculation formula is as follows:

[0104]

[0105] Step 7: Send the minimum resolution difference feature map to the UHRNet decoder. The decoder structure is as follows: Figure 4 As shown, the input of each layer of the decoder is skipped with the output of each layer of the encoder, as shown in Figure 2 The figure shows how the encoder and decoder are connected;

[0106] Step 7.1: Upsample the minimum resolution difference feature map D5' and concatenate it with the difference feature map D4' of the encoder stage 4 to obtain D4", and use D4" and D5' as input to the UHRNet decoder;

[0107] Step 7.2: For each stage of the UHRNet decoder, it consists of only two parallel HRNet branches, and convolution and fusion of two-scale resolution features are performed in each stage. Finally, each stage will output two feature maps with a resolution difference of one times.

[0108] Step 7.3: Between every two adjacent stages of the UHRNet decoder, upsample the output feature map with a larger resolution in the previous stage to obtain a feature map with half the resolution but half the number of channels, and jump-connect it to the feature map of the same resolution size of the encoder, and then use it together with the resolution input before upsampling as the input of the two HR modules of the next stage, and finally obtain 5 feature map outputs of different resolution sizes.

[0109] Step 8: Output the maximum resolution of the above decoder through Figure 6 The edge guidance module shown is then concatenated with itself and fed into the classifier to obtain the final change detection binary image, and deep supervision is added to the remaining small-resolution outputs to accelerate model convergence;

[0110] Step 8.1: For the output maximum resolution feature map D1", pass it through a 1*1 convolution module to obtain the edge prediction map, and then pass it through the softmax function to obtain the output D 1edge , then upsample to the original image size and compare with the edge label to calculate the edge loss. The corresponding loss function is:

[0111]

[0112] Step 8.2: Concatenate the output D1'' and D1'' in the channel dimension, and then send them to another 1*1 convolution module, and get the output D after the softmax function. 1cd Then upsample to the original image size and compare with the change detection label to calculate the change detection loss. The corresponding loss function is:

[0113] Loss cd =-α t (1-p t ) γ log(p t )

[0114] Step 8.3: Using deep supervision, upsample the other resolution outputs directly to the original image size and compare them with the change detection labels to calculate the change detection loss Loss ds The corresponding loss function calculation formula is the same as above; the final loss function calculation formula is as follows:

[0115] Loss=Loss edge +Loss cd+Loss ds

[0116] Step 8.4: The results on the LEVIR-CD test set are as follows Figure 7 As shown in Figure 1, (a) is the image before the change, (b) is the image after the change, (c) is the actual change reference image, and (d) is the detection result image. In Figures (c) and (d), the white part represents the changed area, and the black part represents the unchanged area. The results on our self-made Chengdu Hi-tech Zone target research area test set are shown in Figure 1. Figure 8 As shown, similarly, (a) is the image before the change, (b) is the image after the change, (c) is the change reference image, and (d) is the obtained detection result image. In Figures (c) and (d), the white part represents the changed area, and the black part represents the unchanged area.

[0117] The present invention adopts an encoder-decoder network structure, and combines the HRNet network structure with the U-Net network to build a U-HRNet model. The encoder part of the U-HRNet model is used to extract feature information, and the decoder part of the U-HRNet model is used to reconstruct and restore the change feature information. The U-HRNet structure in the present invention ensures that multi-scale parallelism can be maintained at each stage of the encoder and decoder, and multi-scale fusion is performed to well integrate the detail information contained in the low level and the semantic information contained in the high level, so that the information contained in the obtained feature map is richer. At the same time, the channel and spatial attention modules introduced in the present invention can make the model pay more attention to important information and enhance the feature information of the difference feature map. In addition, the present invention introduces an edge guidance module, and by using the edge label obtained by the change label processing to guide the model to learn edge features, the model can obtain a change detection prediction map with more complete edge information. Through the above method, the present invention effectively improves the problem that the existing change detection method has low detection accuracy on optical remote sensing images with higher resolution, and the edge of the output change detection map is incomplete. Furthermore, the present invention is trained and tested on two different datasets, including a public dataset LEVIR-CD and a self-made dataset, respectively, which also confirms the universality of the proposed method on different datasets.

[0118] The above is only a preferred embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be thought of by a person skilled in the art within the technical scope disclosed by the present invention without creative work should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope defined in the claims.

Claims

1. A method for detecting building changes in optical remote sensing images based on deep learning, comprising the following steps: Step 1: Obtain multispectral images and panchromatic images of the target study area at two different periods, and use ENVI software to perform radiometric calibration, atmospheric correction, and orthorectification on the multispectral images, and radiometric calibration and orthorectification on the panchromatic images. The processed multispectral images and panchromatic images are registered and image fused to obtain the final remote sensing image; Step 2: Use LabCD software to manually mark the changed parts of the buildings in the two remote sensing images of different phases, obtain the corresponding change Label map, and then use the Sobel operator to obtain the edge label of the corresponding change area; Step 3: Preprocess the acquired optical remote sensing image data of two different phases; Step 4: Input the read image T1 before the change and the image T2 after the change into the weight-sharing UHRNet encoder respectively, and extract the feature maps of the image T1 and the image T2 at different levels from stage 1 to stage 5; Step 5: The feature maps of the same stage of image T1 and image T2 are respectively sent to the difference feature extraction module to obtain the difference feature maps of the remote sensing feature images at each stage. The calculation formula is as follows: D i =abs(X i1 ,X i2 ) X i1 11X i2 grainDNBiasC1C2LReσC i D i ' D1”'Loss ds D5'D4”; Where X i1 ,X i2 Denote the feature maps of image T1 and image T2 in the i-th stage, D i represents the output of the i-th stage; Step 6: Send the difference feature maps of each stage to the channel and spatial attention modules respectively. The channel attention module can calculate the importance of each channel and give different attention to each channel; the spatial attention module can assign different weights to different positions of the input data, so that the model pays more attention to important information, thereby obtaining an enhanced difference feature map; Step 7: Feed the minimum resolution difference feature map into the UHRNet decoder and add a skip connection in the decoder; Step 8: The UHRNet decoder output is passed through the edge guidance module and then concatenated with itself and fed into the classifier to obtain the final change detection binary image.

2. The method for detecting building changes in optical remote sensing images based on deep learning according to claim 1, characterized in that: The specific steps of step 1 are as follows: Step 1.1: Perform radiometric calibration on the multispectral image and the panchromatic image first. The digital quantization value DN can be converted into a radiometric brightness value to describe the reflection or emission characteristics of the object, which can be expressed as: Rad=gain*DN+Bias; Where grain is the gain coefficient, DN is the digital quantization value in the image, and Bias is the offset; Step 1.2: Use the FLAASH atmospheric correction tool to perform atmospheric correction on multispectral images; Step 1.3: Use RPC information to orthorectify the multispectral image and panchromatic image, and choose cubic convolution as the resampling method; Step 1.4: Use the NNDiffuse Pan Sharpening method in ENVI to fuse the multispectral image and the panchromatic image to obtain a remote sensing image with both high spatial resolution and spectral resolution.

3. The method for detecting building changes in optical remote sensing images based on deep learning as claimed in claim 1, characterized in that: The specific steps of step 3 are as follows: Step 3.1: After obtaining the multi-temporal optical remote sensing image data, segment it into non-overlapping image blocks with a resolution of 256*256, and divide it into three data sets: training set, validation set, and test set according to the data ratio of 7:1:2; Step 3.2: Read the segmented remote sensing image; Step 3.3: Perform data enhancement operations on the read training set, including random flipping in the horizontal direction, random flipping in the vertical direction, random cropping of the image, and Gaussian blurring of the image.

4. The method for detecting building changes in optical remote sensing images based on deep learning according to claim 1, characterized in that: The specific steps of step 4 are as follows: Step 4.1: For the input feature map X, obtain L feature maps A with a resolution of 1 / 4 of the input through two consecutive two-dimensional convolutions with a step size of 2. The calculation formula is as follows: A=ReLU(BN(Conv(ReLU(BN(Conv(x)))))); Step 4.2: Each stage of the UHRNet encoder consists of only two parallel HR modules, and convolutional features are extracted and two-scale resolution features are fused in each stage. Finally, each stage outputs two feature maps with a resolution difference of one times. Step 4.3: Between every two adjacent stages of the UHRNet encoder, the output feature map with a smaller resolution in the previous stage is downsampled to obtain a feature map with a resolution reduced by half but the number of channels doubled. The feature map is used together with the resolution input before downsampling as the input of the two HR modules in the next stage, and finally 5 feature map outputs with different resolution sizes are obtained.

5. The method for detecting building changes in optical remote sensing images based on deep learning as claimed in claim 1, characterized in that: The specific steps of step 6 are as follows: Step 6.1: D i It is sent to the channel attention module CAM, where the maximum pooling is applied first, and then the attention weight is calculated through two fully connected layers and a sigmoid activation function. The reconstructed attention weight vector is multiplied element-by-element with the input feature map to obtain the output of the CAM. The calculation formula is as follows: C i =σ(C2(LRe(C1(max(D i ))))); Where C1 and C2 represent fully connected layers, LRe represents the Leaky-ReLU activation function, σ represents the sigmoid activation function, and C i represents the channel attention weight of the i-th stage, D i represents the difference feature map of the i-th stage, D ic Added the difference feature map of channel attention for the i-th layer; Step 6.2: D i It is also sent to the spatial attention module SAM, which condenses the spatial information of the feature map through max pooling and average pooling, and then uses the convolution layer and sigmoid activation function to calculate the spatial attention weight; the output of SAM is obtained by multiplying the obtained spatial attention weight with the input feature map element by element. The calculation formula is as follows: S i =σ(Conv([max(D i )+avg(D i )])); Where S i represents the channel attention weight of the i-th stage, D i represents the difference feature map of the i-th stage, D is Added the difference feature map of spatial attention for the i-th layer; Step 6.3: Add the outputs of the channel and spatial attention modules to get the feature output D i ', the calculation formula is as follows:

6. The method for detecting building changes in optical remote sensing images based on deep learning as claimed in claim 1, characterized in that: The specific steps of step 7 are as follows: Step 7.1: Upsample the minimum resolution difference feature map D5' and concatenate it with the difference feature map D4' of the encoder stage 4 to obtain D4", and use D4" and D5' as input to the UHRNet decoder; Step 7.2: For each stage of the UHRNet decoder, it consists of only two parallel HRNet branches, and convolution and fusion of two-scale resolution features are performed in each stage. Finally, each stage will output two feature maps with a resolution difference of one times. Step 7.3: Between every two adjacent stages of the UHRNet decoder, upsample the output feature map with a larger resolution in the previous stage to obtain a feature map with half the resolution but half the number of channels, and jump-connect it to the feature map of the same resolution size of the encoder, and then use it together with the resolution input before upsampling as the input of the two HR modules of the next stage, and finally obtain 5 feature map outputs of different resolution sizes.

7. The method for detecting building changes in optical remote sensing images based on deep learning as claimed in claim 1, characterized in that: The specific steps of step 8 are as follows: Step 8.1: For the output maximum resolution feature map D1", pass it through a 1*1 convolution module to obtain the edge prediction map, and then pass it through the softmax function to obtain the output D 1edge , then upsample to the original image size and compare with the edge label to calculate the edge loss. The corresponding loss function is: Step 8.2: Concatenate the output D1'' and D1'' in the channel dimension, and then send them to another 1*1 convolution module, and get the output D after the softmax function. 1cd Then upsample to the original image size and compare with the change detection label to calculate the change detection loss. The corresponding loss function is: Loss cd =-α t (1-p t ) γ log(p t ); Step 8.3: Using deep supervision, upsample the other resolution outputs directly to the original image size and compare them with the change detection labels to calculate the change detection loss Loss ds ; The corresponding loss function calculation formula is the same as above; the final loss function calculation formula is as follows: Loss=Loss edge +Loss cd +Loss ds 。

Citation Information

Patent Citations

  • Optical remote sensing image change detection method based on adaptive fusion NestedUNet

    CN115393718A

  • High-resolution optical remote sensing image building change detection method

    CN115471467A

  • High-resolution remote sensing image building change detection method and device and electronic equipment

    CN116071645A

  • Wheat planting area identification method and system, electronic equipment and storage medium

    CN116844053A

  • Method for extracting building change area in double-time-phase remote sensing image based on twinborn mixed attention mechanism and multi-scale feature fusion

    CN118212532A

Cited By

  • Construction process monitoring method based on construction remote sensing image analysis

    CN120046872A

  • A construction process monitoring method based on the analysis of construction remote sensing images

    CN120046872B

  • Water body extraction method and device based on multi-source multi-scale optical remote sensing data

    CN120411810A