A lightweight network-based building change detection method

Abstract

This invention proposes a building change detection method based on a lightweight network, comprising: acquiring remote sensing image pairs of the area to be detected at different time phases; extracting multi-level features from the remote sensing image pairs respectively; calculating a building change detection result map based on the fusion of the multi-level features of the images; performing post-processing on the building change detection result; and finally, vectorizing the post-processed result to obtain the final vector result of building change detection. This invention uses a lightweight network to extract building features, reducing the number of network parameters and computational cost, ensuring a smaller network size and faster running speed; simultaneously, it utilizes deconvolution to dynamically learn weight parameters during the feature fusion stage, reducing the loss of useful information during the size restoration stage, thereby ensuring the accuracy of building change detection.

Description

Technical Field

[0001] This invention belongs to the field of building change detection technology, specifically relating to a building change detection method based on a lightweight network. Background Technology

[0002] In recent years, deep learning-based methods for detecting building changes have developed rapidly, with convolutional neural networks providing new ideas for optimizing building change detection algorithms due to their powerful feature extraction capabilities.

[0003] Existing convolutional neural network-based building change detection methods can be categorized into two main types based on their design strategies: classification-then-comparison and direct comparison. The classification-then-comparison method first uses a convolutional neural network to extract features or building results from images before and after the change, and then compares these features or results. While the feature extraction method can automatically learn deep features, it is essentially still a pixel-by-pixel analysis, making it difficult to address pseudo-changes caused by different shooting angles. The method that outputs building results first requires both change labels and dual-temporal semantic labels, but such datasets are currently scarce. Direct classification methods can be further divided into single-stream direct classification and two-stream direct classification based on their network framework design. In a single-stream structure, two images are directly stitched together by channel or the difference image processed from the two images is input into a semantic segmentation network to obtain the change results. Although this type of method utilizes rich features across multiple scales and levels, it neglects the independent features of individual images during the encoding and decoding process, such as the boundary integrity of individual images and the compactness of the building's interior, leading to boundary errors and voids in the detection results. In a two-stream structure, two identical feature extractors are typically used to extract features from the two images respectively, and then the feature fusion network is used to obtain the change results. This type of method performs well in building change detection, but the network size is huge. When using a network structure with "very deep" features, it will lead to problems such as huge computational cost. Summary of the Invention

[0004] The technical problem this invention aims to solve is the high computational cost associated with building change detection methods that use complex deep networks and multi-scale fusion. It proposes a building change detection method based on lightweight networks. This method can reduce the number of model parameters and computational load while maintaining the accuracy of change detection.

[0005] This invention employs a change detection network with a dual-stream structure, using MobilenetV2 as a feature extractor to extract multi-level features from two phases of images, then fuses the multi-level features, and finally obtains the change detection result.

[0006] The technical solution adopted in this invention is as follows:

[0007] A method for detecting building changes based on lightweight networks, comprising the following steps:

[0008] Step 1) Acquire remote sensing images of the area to be detected, including images A and B acquired at times t1 and t2 respectively;

[0009] Step 2) Extract features from building change detection images A and B, obtaining feature AF for image A and feature BF for image B respectively:

[0010] A weight-sharing dual-stream network is used to extract features from change detection image pairs A and B, respectively. The feature extractor in the dual-stream network branch is the lightweight network MobileNetV2. The fourth and seventeenth layers of MobileNetV2 are used as the low-level and high-level features of change detection image pairs A and B, respectively. Therefore, the features of building change detection image pairs A and B are AF = {A...} Fi |i=4,17} and BF={B Fi |i = 4, 17};

[0011] Step 3) Calculate the changes in buildings in images A and B:

[0012] 3.1) Fuse the low-level and high-level features of images A and B obtained in step 2) respectively to obtain the fused feature F. l and F h It is expressed as follows:

[0013] F l =conv k＝3 (conv k＝3 (concat(A F4 B F4 )))#(1)

[0014] F h =conv k＝3 (conv k＝3 (concat(A F17 B F17 )))#(2)

[0015] Where, conv k＝3 It is a convolution operation with a 3x3 kernel, and concat is a channel concatenation operation;

[0016] 3.2) The advanced fusion feature F described in step 3.1) h Multi-scale features are further extracted using spatial pyramid pooling and upsampled to the low-level fused feature F. l Consistent dimensions, thus obtaining feature F′ h :

[0017] F′ h =Upsampling bilinear (ASPP(F h ))#(3)

[0018] Among them, Upsampling bilinear This indicates an upsampling operation using bilinear interpolation, while ASPP represents a spatial pyramid pooling operation.

[0019] 3.3) The low-level fusion feature F described in step 3.1) l conv using 1x1 convolution k＝1 To obtain F′ by dimensionality increase l , means as follows:

[0020] F′ l =conv k＝1 (F l )#(4)

[0021] 3.4) The feature F′ described in step 3.3) l Compared with the feature F′ described in step 3.2) h Further fusion yields F″:

[0022] F″ = conv k＝3 (conv k＝3 (concat(F′ l F′ h )))#(5)

[0023] Then, two sets of deconvolution and convolution operations are used to process F″ to obtain the feature F′:

[0024] F′=conv k＝3 (deconv k＝2，s＝2 (conv k＝3 (deconv k＝2，s＝2 (F″))))#(6)

[0025] Among them, deconv k＝2，s＝2 This represents a deconvolution operation with a 2x2 kernel and a step count of 2;

[0026] 3.5) Classify the feature F′ described in step 3.4) using a 1x1 convolution to obtain the predicted class probability result P:

[0027] P = conv k＝1 (F′)#(7)

[0028] The loss function used in the change detection network cd As shown below:

[0029] losscd =loss ce +loss dice #(8)

[0030] Where, loss ce The loss function is a type II cross-entropy loss function. dice The Dice loss function;

[0031] 3.6) Binarize the predicted probability described in step 3.5) to obtain the building change detection result image I. The binarization result I(p) of any pixel is calculated as follows:

[0032]

[0033] Where, p i Let i represent the class probability of any pixel in the predicted class probability result P, where i represents the class. Class 0 represents a pixel that has not changed, and class 1 represents a building that has changed. In the building change detection result image I, 255 represents a building that has changed, and 0 represents an area that has not changed.

[0034] Step 4) Post-process the building change detection result image I obtained in step 3.6), including removing small connected areas and filling holes, to obtain the final change detection result image R;

[0035] Step 5) Vectorize the change detection result map R described in Step 4) to obtain the final vector result of the changed building.

[0036] The beneficial effects of this invention are as follows:

[0037] 1) This invention uses a lightweight network in the feature extraction stage, which reduces the number of network parameters and computational cost, ensuring that the network has a smaller size and faster running speed.

[0038] 2) In the feature fusion stage, the present invention utilizes deconvolution to dynamically learn weight parameters, thereby reducing the loss of useful information in the size recovery stage and ensuring the accuracy of building change detection. Attached Figure Description

[0039] Figure 1 This is a flowchart of the present invention;

[0040] Figure 2 This is an image before the changes in this embodiment of the invention;

[0041] Figure 3 This is a modified image from an embodiment of the present invention;

[0042] Figure 4 This is a network structure diagram in an embodiment of the present invention;

[0043] Figure 5 It is an image pair Figure 2 and Figure 3 The change detection prediction map predicted using the method of the embodiments of the present invention shows that white represents the buildings that have changed and black represents the background. Detailed Implementation

[0044] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0045] A building change detection method based on lightweight networks includes the following steps:

[0046] Step 1: Acquire remote sensing images of the area to be detected. The remote sensing images include images A and B acquired at times t1 and t2, respectively. In this embodiment of the invention, image A is as follows: Figure 2 As shown, image B is as follows Figure 3 As shown;

[0047] Step 2: Extract features from building change detection images A and B, obtaining feature AF for image A and feature BF for image B respectively:

[0048] A weight-sharing dual-stream network is used to extract features from change detection image pairs A and B, respectively. The feature extractor in the dual-stream network branch is the lightweight network MobileNetV2. The fourth and seventeenth layers of MobileNetV2 are used as the low-level and high-level features of change detection image pairs A and B, respectively. Therefore, the features of building change detection image pairs A and B are AF = {A...} Fi |i=4,17} and BF={B Fi |i = 4, 17};

[0049] Step 3: Calculate the changes in buildings in images A and B:

[0050] (31) The low-level and high-level features of images A and B described in step 2 are fused to obtain the fused feature F. l and F h It is expressed as follows:

[0051] F l =conv k＝3 (conv k＝3 (concat(A F4 B F4 )))#(1)

[0052] F h =conv k＝3 (conv k＝3 (concat(A F17 B F17 )))#(2)

[0053] Where, conv k＝3 It is a convolution operation with a 3x3 kernel, and concat is a channel concatenation operation;

[0054] (32) The advanced fusion feature F described in step (31) h Multi-scale features are further extracted using spatial pyramid pooling and upsampled to the low-level fused feature F. l Consistent dimensions, thus obtaining feature F′ h :

[0055] F′ h =Upsampling bilinear (ASPP(F h ))#(3)

[0056] Among them, Upsampling bilinear This indicates an upsampling operation using bilinear interpolation, while ASPP represents a spatial pyramid pooling operation.

[0057] (33) The low-level fusion feature F described in step (31) l conv using 1x1 convolution k＝1 To obtain F′ by dimensionality increase l , means as follows:

[0058] F′ l =conv k＝1 (F l )#(4)

[0059] (34) The feature F′ described in step (33) l With the feature F′ described in step (32) h Further fusion yields F″:

[0060] F″ = conv k＝3 (conv k＝3 (concat(F′ l F′ h )))#(5)

[0061] Then, two sets of deconvolution and convolution operations are used to process F″ to obtain the feature F′:

[0062] F′=conv k＝3 (deconv k＝2，s＝2 (conv k＝3 (deconv k＝2，s＝2 (F″))))#(6)

[0063] Among them, deconv k＝2，s＝2 This represents a deconvolution operation with a 2x2 kernel and a step count of 2;

[0064] (35) Classify the feature F′ described in step (34) using a 1x1 convolution to obtain the predicted class probability result P:

[0065] P = conv k＝1 (F′)#(7)

[0066] The loss function used in the change detection network cd As shown below:

[0067] loss cd =loss ce +loss dice #(8)

[0068] Where, loss ce The loss function is a type II cross-entropy loss function. dice The Dice loss function;

[0069] (36) Binarize the predicted probability described in step (35) to obtain the building change detection result image I. The binarization result I(p) of any pixel is calculated as follows:

[0070]

[0071] Where, p i Let represent the class probability of any pixel in the predicted class probability result P, where i represents the class. Class 0 indicates no change in the pixel, and class 1 indicates a changing building. In the building change detection result image I, 255 represents a changing building, and 0 represents an unchanging area. The change detection network structure in this implementation case is as follows: Figure 4 As shown.

[0072] Step 4: Post-process the building change detection result image I described in step (36), including removing small connected areas and filling holes, to obtain the final change detection result image R. The final building change detection result in this example is as follows: Figure 5 As shown.

[0073] Step 5: Vectorize the change detection result image R described in Step 4 to obtain the final vector result of the changed building.

[0074] The above is merely a description of embodiments of the present invention, but the scope of protection of the present invention should not be considered as limited to the specific forms described in the embodiments. The scope of protection of the present invention also extends to equivalent technical means that can be conceived by those skilled in the art based on the concept of the present invention.

Claims

1. A method for detecting building changes based on lightweight networks, characterized in that, Includes the following steps: Step 1) Acquire remote sensing images of the area to be detected, including images A and B acquired at times t1 and t2 respectively; Step 2) Extract features from building change detection images A and B, obtaining feature AF for image A and feature BF for image B respectively: A weight-sharing dual-stream network is used to extract features from change detection image pairs A and B, respectively. The feature extractor in the dual-stream network branch is the lightweight network MobileNetV2. The fourth and seventeenth layers of MobileNetV2 are used as the low-level and high-level features of change detection image pairs A and B, respectively. Therefore, the features of building change detection image pairs A and B are AF = {A...} Fi |i=4,17} and BF={B Fi |i=4,17}; Step 3) Calculate the changes in buildings in images A and B: 3.1) Fuse the low-level and high-level features of images A and B obtained in step 2) respectively to obtain the fused feature F. l and F h It is expressed as follows: F l =conv k＝3 (conv k＝3 (concat(A F4 ,B F4 )))#(1) F h =conv k＝3 (conv k＝3 (concat(A F17 ,B F17 )))#(2) Where, conv k＝3 It is a convolution operation with a 3x3 kernel, and concat is a channel concatenation operation; 3.2) The advanced fusion feature F described in step 3.1) h Multi-scale features are further extracted using spatial pyramid pooling and upsampled to the low-level fused feature F. l Consistent dimensions, thus obtaining feature F' h : F′ h =Upsampling bilinear (ASPP(F h ))#(3) Among them, Upsampling bilinear This indicates an upsampling operation using bilinear interpolation, while ASPP represents a spatial pyramid pooling operation. 3.3) The low-level fusion feature F described in step 3.1) l conv using 1x1 convolution k＝1 To obtain F' by dimensional expansion l , means as follows: F′ l =conv k＝1 (F l )#(4) 3.4) The feature F' described in step 3.3) l With the feature F' described in step 3.2) h Further fusion yields F": F”=conv k＝3 (conv k＝3 (concat(F' l ,F′ h )))#(5) Then, two sets of deconvolution and convolution operations are used to process F” to obtain the feature F’: F=conv k＝3 (deconv k＝2,s＝2 (conv k＝3 (deconv k＝2,s＝2 (F″))))#(6) in, deconv k＝2,s＝2 This represents a deconvolution operation with a 2x2 kernel and a step count of 2; 3.5) Classify the feature F' described in step 3.4) using a 1x1 convolution to obtain the predicted class probability result P: P=conv k＝1 (F′)#(7) The loss function used in the change detection network cd As shown below: loss cd =loss ce +loss dice #(8) Where, loss ce The loss function is a type II cross-entropy loss function. dice The Dice loss function; 3.6) Binarize the predicted probability described in step 3.5) to obtain the building change detection result image I. The binarization result I(p) of any pixel is calculated as follows: Where, p i This represents the class probability of any pixel in the predicted class probability result P, where i represents the class. Class 0 indicates that the pixel has not changed, and class 1 indicates that the building has changed. In the building change detection result image I, 255 represents the building that has changed, and 0 represents the area that has not changed. Step 4) Post-process the building change detection result image I obtained in step 3.6), including removing small connected areas and filling holes, to obtain the final change detection result image R; Step 5) Vectorize the change detection result map R described in Step 4) to obtain the final vector result of the changed building.

Images