A multi-scale feature guided building change detection method

Abstract

The application provides a building change detection method under multi-scale feature guidance, and comprises the following steps: training a deep convolutional neural network for building feature extraction by using a building data set; extracting multi-scale building features of a change detection image pair in a building change detection data set; training and optimizing a change detection model based on multi-scale feature fusion; detecting changed buildings in the image pair of the same place at different times to obtain a prediction probability graph of a change result and perform binary processing; and finally, the result graph can be vectorized to obtain a vector file containing the changed buildings. The deep convolutional neural network is fully utilized to extract building features, so that the interference of the background in the change detection process is reduced; meanwhile, the multi-scale building features can guide the network to focus on building recognition, so that the integrity of the predicted building is ensured, the generation of false changes is reduced, and the precision of the building change detection is improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of building change detection, and particularly relates to a building change detection method guided by multi-scale features. BACKGROUND

[0002] With the continuous acceleration of urbanization, the ground structure of urban areas is undergoing earth-shaking changes. Buildings, as one of the main ground objects in urban areas, are the most likely to change and most need to be updated in urban geographic databases. Timely and accurate acquisition of building change information can provide important data support for the city planning industry.

[0003] Traditional change detection methods mainly include direct comparison method, image transformation method and post-classification comparison method. Among them, the direct comparison method compares the spectral information in two images to determine the change area, and then determines the change type by manual method, mainly including image algebra method and image regression method; the image transformation method first converts the image into a specific representation, and then analyzes the specific representation, mainly including principal component analysis, hue-saturation-value transformation and canonical correlation analysis method; the post-classification comparison method classifies two images respectively, and then compares the classification results to obtain the change result.

[0004] In recent years, with the rapid development of deep learning technology, this technology has been introduced into the research of building change detection. The existing change detection methods based on deep neural network can be divided into pixel-based and object-based methods according to the processing unit. The pixel-based method compares the images of different periods pixel by pixel on the basis of image registration to analyze the change result, which is easily affected by image registration, radiation correction and other factors; the object-based change detection method fully utilizes various feature information of objects for change detection analysis. However, the current building change detection method based on deep neural network still has the problems of incomplete change building detection and pseudo change. SUMMARY

[0005] The technical problem to be solved by the present application is that the existing deep neural network change detection method generally has the problem of incomplete change detection and pseudo change due to insufficient building feature extraction. The present application provides a building change detection method guided by multi-scale features. This method can effectively utilize the characteristics of building objects, thereby improving the accuracy of change detection.

[0006] The present application uses a deep convolutional network to obtain the multi-scale features of buildings, then inputs the building fusion features of two images into a multi-scale fusion change detection network, obtains a change prediction map through the change detection network, and finally performs vectorization.

[0007] The technical scheme adopted by the present application is a building change detection method under multi-scale feature guidance, which comprises the following steps:

[0008] Step 1, a deep convolutional neural network for extracting building features is trained to obtain a multi-scale building feature extraction model BM:

[0009] The deep neural network is trained using a building dataset, wherein the building dataset comprises satellite images and corresponding building label maps;

[0010] The deep convolutional neural network for building extraction has a "coding-decoding" structure of U-Net, the encoder mainly consists of five groups of convolutions, and each group of convolutions contains four downsampling operations, the decoder contains five groups of convolutions corresponding to the encoder, each group of convolutions starts with a channel splicing operation corresponding to the encoder at the corresponding scale, and each group of convolutions contains an upsampling operation, the features output by the five groups of convolutions in the decoder are respectively denoted as F1, F2, F3, F4 and F5, and after the output feature of the last layer of the decoder, a 1x1 convolution and a Sigmod activation function are used to complete classification. The loss function used by the network is the binary cross-entropy, which is shown in the following formula:

[0011] loss bce =-Yb*log(Pb)-(1-Yb)*log(1-Pb) (1)

[0012] Wherein, Yb is a building label map, and Pb is a predicted probability map.

[0013] Step 2, multi-scale building features of the change detection image pair in the building change detection dataset are extracted:

[0014] The building change detection dataset comprises image pairs of the same place at different times and corresponding building change label maps, and the change detection image pair feature extraction method is as follows: the change detection image pairs A and B of the same place at different imaging times are respectively input into the multi-scale building feature extraction model BM trained in step 1, and the five groups of convolutions F1, F2, F3, F4 and F5 of the decoder described in step 1 are calculated as the multi-scale building features AF={A Fi |i=1,2,3,4,5} and BF={B Fi |i=1,2,3,4,5} of the change detection image pairs A and B.

[0015] Step 3, a change detection model based on multi-scale feature fusion is trained and optimized:

[0016] (31) using the multi-scale building features of the change detection image pair in the building change detection data set obtained in step 2 and the label map of the change detection image pair in step 2 for training of a change detection network, the change detection network comprising a feature fusion module and a change detection module;

[0017] In the feature fusion module, the multi-scale building features AF and BF obtained in step 2 are fused layer by layer in the channel splicing manner according to the scale to obtain five groups of fused features MF = {M Fi |i = 1, 2, 3, 4, 5};

[0018] The fused features MF = {M Fi |i = 1, 2, 3, 4, 5} and the label map of the change detection image pair in step 2 are used for training of the change detection module;

[0019] The change detection module comprises the decoder part described in step 1, mainly including a convolutional layer corresponding to the input multi-scale features and a channel splicing operation of the corresponding scale, and after the output feature of the last layer of the decoder, a 1x1 convolutional layer and a Sigmod activation function are used to convert the final feature map into a building change detection prediction probability map Pc, and the loss function loss cd of the change detection network is as follows:

[0020] loss cd = α * loss bce + (1-α) * loss dice (2)

[0021] wherein, α represents a weight, loss bce is the binary cross-entropy described in step 1, and loss dice is a Dice loss function, and the formula is as follows:

[0022]

[0023] wherein, Yc is a change detection label map, and Pc is a building change detection prediction probability map.

[0024] (32) selecting a building change detection model CM:

[0025] The change detection model with the highest F1 value on the validation set is selected as the prediction model, and the calculation method of the F1 value is as follows:

[0026]

[0027] wherein, Precision is the precision, and Recall is the recall, and the calculation methods thereof are as follows:

[0028]

[0029]

[0030] Wherein, TP is the number of correctly predicted changed building pixels, FP is the number of incorrectly predicted changed building pixels, FN is the number of incorrectly predicted background pixels;

[0031] Step 4, detecting changed buildings; adopting a picture expression form or a vector expression form;

[0032] The picture expression form comprises:

[0033] (41) inputting a to-be-detected image pair of the same place at different time phases into the multi-scale building feature extraction model BM obtained in step 1, calculating the multi-scale building features AF and BF of the image pair according to the method in step 2, and inputting the multi-scale building features AF and BF into the change detection model CM obtained in step (32), so that a prediction probability graph P of building change detection can be obtained;

[0034] (42) binarizing the prediction probability graph P obtained in step (41) to obtain a building change detection result graph I, and the binarization formula is as follows:

[0035]

[0036] Wherein, 255 represents changed buildings, 0 represents unchanged areas, tau represents a threshold value of binarization, and p represents any pixel value in the prediction probability graph P.

[0037] The vector expression form comprises: vectorizing the building change detection result graph I to obtain a vector file V of changed buildings I .

[0038] The present application has the following advantages:

[0039] 1) The present application fully utilizes a deep convolutional neural network to extract semantic features of buildings in a changed detection area, so that the interference of background areas in the change detection process is reduced;

[0040] 2) The present application fully utilizes semantic features of buildings, guides the network to focus on building identification, ensures the integrity of predicted buildings, reduces the generation of pseudo changes, and thus improves the precision of building change detection. BRIEF DESCRIPTION OF DRAWINGS

[0041] Figure 1 It is a flowchart of a building change detection method under multi-scale feature guidance.

[0042] Figure 2This is a flowchart of the multi-scale building feature extraction network in an embodiment of the present invention.

[0043] Figure 3 This is a flowchart of the multi-scale fusion change detection network in an embodiment of the present invention.

[0044] Figure 4 This is the image before the changes in this embodiment of the invention.

[0045] Figure 5 This is a modified image in an embodiment of the present invention.

[0046] Figure 6 It is an image pair Figure 4 and Figure 5 The change detection prediction map predicted using the method of the embodiments of the present invention shows that white represents the buildings that have changed and black represents the background. Detailed Implementation

[0047] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0048] The technical solution of this invention is a method for detecting building changes guided by multi-scale features, comprising the following steps:

[0049] Step 1: Train a deep convolutional neural network to extract building features, resulting in a multi-scale building feature extraction model BM.

[0050] A deep neural network was trained using a building dataset, which included satellite imagery and corresponding building label maps.

[0051] The deep convolutional neural network for building extraction has a U-Net "encoder-decoder" structure. The encoder mainly consists of five sets of convolutions, with four downsampling operations between each set. In this embodiment, the five sets of convolutions in the encoder use ResNet50's conv1, conv2_x, conv3_x, conv4_x, and conv5_x layers. The decoder contains five sets of convolutions corresponding to the encoder. Each set of convolutions begins with a channel concatenation operation at the scale corresponding to the encoder, and each set of convolutions includes an upsampling operation. In this embodiment, the upsampling operation is implemented using deconvolution. The features output by the five sets of convolutions in the decoder are denoted as F1, F2, F3, F4, and F5, respectively. After the output features of the last layer of the decoder, a 1x1 convolution and a Sigmoid activation function are used to complete the classification. The building feature extraction network structure of this embodiment is as follows: Figure 2 As shown. The network uses binary classification cross-entropy as its loss function, as shown in the formula below:

[0052] loss bce=-Yb*log(Pb)-(1-Yb)*log(1-Pb) (1)

[0053] Where Yb is the building label map and Pb is the prediction probability map.

[0054] Step 2: Extract multi-scale building features from change detection image pairs in the building change detection dataset:

[0055] The building change detection dataset includes image pairs of the same location at different times and corresponding building change label maps. The feature extraction method for the change detection image pairs is as follows: The change detection image pairs A and B at the same location but different imaging times are input into the multi-scale building feature extraction model BM trained in step 1. The five sets of convolutions F1, F2, F3, F4, and F5 of the decoder described in step 1 are calculated respectively, serving as the multi-scale building features AF = {A} of the change detection image pairs A and B. Fi |i=1,2,3,4,5} and BF={B Fi |i=1,2,3,4,5}.

[0056] Step 3: Train and optimize the change detection model based on multi-scale feature fusion:

[0057] (31) Use the multi-scale building features of the change detection image pairs in the building change detection dataset obtained in step 2 and the label map of the change detection image pairs in step 2 for training the change detection network. The change detection network includes a feature fusion module and a change detection module.

[0058] In the feature fusion module, a channel splicing method is used to fuse the multi-scale building features AF and BF obtained in step 2 layer by layer according to scale, resulting in five sets of fused features MF = {M Fi In this embodiment, channel splicing is represented by concat; |i=1,2,3,4,5}.

[0059] The fusion feature MF = {M Fi The label maps of the change detection image pairs described in step 2 and |i=1,2,3,4,5} are used for training the change detection module;

[0060] The change detection module includes the decoder section described in step 1, mainly comprising convolutional layers corresponding to the input multi-scale features and channel concatenation operations at the corresponding scales. After the output features of the last layer of the decoder, a 1x1 convolution and a Sigmoid activation function are used to transform the final feature map into a predicted probability map Pc for building change detection. The network structure of this embodiment is as follows: Figure 3 The change detection network uses the loss function... cd As shown below:

[0061] loss cd =α*loss bce +(1-α)*loss dice (2)

[0062] Where α represents the weight, and in this embodiment of the invention, α is set to 0.5, loss bce It is the binary classification cross-entropy mentioned in step 1, loss dice The Dice loss function is defined by the following formula:

[0063]

[0064] Where Yc is the change detection label map, and Pc is the predicted probability map for building change detection.

[0065] (32) Select the building change detection model CM:

[0066] The change detection model with the highest F1 score on the validation set is selected as the prediction model. The F1 score is calculated as follows:

[0067]

[0068] Where Precision is the accuracy rate and Recall is the recall rate, their calculation methods are as follows:

[0069]

[0070]

[0071] Where TP is the number of correctly predicted changing building pixels, FP is the number of incorrectly predicted changing building pixels, and FN is the number of incorrectly predicted background pixels;

[0072] Step 4: Detect changes in buildings:

[0073] (41) Input the image pairs to be detected at the same location at different times into the multi-scale building feature extraction model BM obtained in step 1. In this embodiment, image pairs at the same location at different times are selected. Figure 4 and Figure 5 Calculate the multi-scale building features AF and BF of the image pair according to the method described in step 2, and input the multi-scale building features AF and BF into the change detection model CM obtained in step (32), thereby obtaining the predicted probability map P for building change detection.

[0074] (42) Binarize the predicted probability map P obtained in step (41) to obtain the building change detection result map I. The binarization formula is as follows:

[0075]

[0076] Where 255 represents the changing building, 0 represents the unchanged area, τ represents the binarization threshold (in this embodiment, τ is set to 0.5), and p represents any pixel value in the prediction probability map P. The binarization result of the building change detection in this embodiment is as follows: Figure 6 As shown.

[0077] Vectorize the building change detection result image I to obtain the vector file V of the changed building. I .

[0078] The effectiveness of the method of the present invention will be further explained below with reference to image pairs in the embodiments of the present invention. The prediction results of image pairs according to the embodiments of the present invention... Figure 6 As can be seen, the detection method of this invention can identify most of the changed building results. The identified building objects are complete, with clear outlines and well-defined boundaries, exhibiting few false changes. It can also identify small buildings with incomplete edge areas in the image. Furthermore, four indicators—F1, Precision, Recall, and OA—are used for quantitative evaluation. The formula for calculating the overall precision OA is as follows:

[0079]

[0080] Here, TN is the number of correctly predicted background pixels.

[0081] Prediction results in the embodiments of the present invention Figure 6 The precision was 94.55%, recall was 89.42%, F1 score was 91.91%, and OA was 95.53%. In summary, the method in this embodiment enhances the sensitivity to the semantic features of building objects, enabling it to keenly capture differences in changing regions and effectively suppress information in non-changing regions, thereby improving the accuracy of building change detection. Furthermore, by leveraging prior information from building semantic features guided by these features, the detection performance is further enhanced, resulting in more complete buildings with clearer outlines and more defined boundaries in the recognition results, effectively improving the misclassification and omission issues of existing deep neural network post-classification.

[0082] The above is merely a description of embodiments of the present invention, but the scope of protection of the present invention should not be considered as limited to the specific forms described in the embodiments. The scope of protection of the present invention also extends to equivalent technical means that can be conceived by those skilled in the art based on the concept of the present invention.

Claims

1. A method for building change detection under multi-scale feature guidance, characterized in that, Comprising the following steps: Step 1, training a deep convolutional neural network for extracting building features to obtain a multi-scale building feature extraction model BM: Training a deep neural network using a building dataset, wherein the building dataset comprises satellite images and corresponding building label maps; The deep convolutional neural network for building extraction has a "coding-decoding" structure of U-Net, the encoder mainly consists of five groups of convolutions, and there is a downsampling operation between each group of convolutions, the decoder contains five groups of convolutions corresponding to the encoder, and each group of convolutions has a channel splicing operation corresponding to the encoder at the beginning, and each group of convolutions contains an upsampling operation, the features output by the five groups of convolutions in the decoder are denoted as F1, F2, F3, F4 and F5 respectively, and after the last layer of the decoder outputs the feature, a 1x1 convolution and a Sigmod activation function are used to complete the classification, the loss function used by the network is binary cross-entropy, and the formula is as follows: loss bce = -Yb*log(Pb) - (1 - Yb)*log(1 - Pb) (1) Wherein, Yb is a building label map, and Pb is a prediction probability map; Step 2, extracting multi-scale building features of the change detection image pair in the building change detection dataset: The building change detection dataset comprises image pairs of the same place at different time phases and corresponding building change label images. The feature extraction method of the change detection image pair is as follows: the change detection image pair A and B of the same place at different imaging times are respectively input into the multi-scale building feature extraction model BM trained in step 1, and five groups of convolutions F1, F2, F3, F4 and F5 of the decoder in step 1 are calculated as the multi-scale building features AF={A Fi |i=1,2,3,4,5} and BF={B Fi |i=1,2,3,4,5} of the change detection image pair A and B. Step 3, training and optimizing the change detection model based on multi-scale feature fusion: (31) using the multi-scale building features of the change detection image pair in the building change detection dataset obtained in step 2 and the label map of the change detection image pair in step 2 for training of the change detection network, the change detection network comprising a feature fusion module and a change detection module; In the feature fusion module, the channel splicing method is adopted, and the multi-scale building features AF and BF obtained in step 2 are fused layer by layer according to the scale to obtain five groups of fused features MF = {M Fi i = 1, 2, 3, 4, 5}. The fused features MF = {M Fi The label maps of the change detection image pairs described in step 2 are used for training of the change detection module. The change detection module includes the decoder part described in step 1, mainly including a convolutional layer corresponding to the input multi-scale feature and a channel concatenation operation of the corresponding scale. After the output feature of the last layer of the decoder, a 1x1 convolution and a Sigmod activation function are used to convert the final feature map into a prediction probability map Pc of building change detection. The loss function loss adopted by the change detection network is as follows: cd As shown below: loss cd = a * loss bce + (1 - a) * loss dice (2) wherein a represents a weight, loss bce is the binary cross-entropy described in step 1, loss dice is the Dice loss function, whose formula is as follows: Wherein, Yc is a change detection label map, and Pc is a prediction probability map of building change detection; (32) selecting a building change detection model CM: Select the change detection model with the highest F1 value on the validation set as the prediction model, and the calculation method of F1 value is as follows: Wherein, Precision is the precision, and Recall is the recall, and their calculation methods are as follows: Wherein, TP is the number of pixels of the predicted correct change building, FP is the number of pixels of the predicted incorrect change building, and FN is the number of pixels of the predicted incorrect background; Step 4, detecting the changed building: adopting a picture expression form or a vector expression form; The picture expression form comprises: (41) inputting the same place and different time image pairs to be detected into the multi-scale building feature extraction model BM obtained in step 1, calculating the multi-scale building features AF and BF of the image pairs according to the method described in step 2, and inputting the multi-scale building features AF and BF into the change detection model CM obtained in step (32), thereby obtaining a prediction probability map P of building change detection; (42) binarizing the prediction probability map P obtained in step (41) to obtain a building change detection result map I, and the binarization formula is as follows: Wherein, 255 represents the changed building, 0 represents the unchanged area, τ represents the threshold value of binarization, and p represents any pixel value in the prediction probability map P; The vector expression form includes: vectorizing the building change detection result image I to obtain a vector file V of the changed building I .

Images