A center net-based remote sensing image building change detection model and algorithm

Abstract

The present invention discloses a remote sensing image building change detection model and algorithm based on CenterNet. The model includes a feature extraction module, a building semantic segmentation module, and a change detection module. The feature extraction module is composed of two CenterNet networks with shared parameters. The Encoder module of the CenterNet network adopts a ResNet18 network pre-trained based on an ImageNet data set as a backbone network. Two images are respectively input into the feature extraction module to obtain corresponding feature maps. The building semantic segmentation module extracts feature maps from the two images, performs spatial pyramid pooling, and then uses a softmax classifier to predict pixel by pixel to form a semantic segmentation map of the changed area. The change detection module calculates the absolute difference between the feature maps extracted from the two images to obtain a difference feature map. After spatial pyramid pooling, a Sigmoid classifier is used to predict pixel by pixel to form a building change map. The loss function of the model is a joint cost loss function L=L ss1 +L ss2 +2*L wbce , where L ss1 , L ss2 are the cross entropy losses between the predicted value and the true value; L wbce is the weighted binary cross entropy loss between the predicted value and the true value.

Description

Technical Field

[0001] The present invention relates to the technical field of building change detection, and in particular to a CenterNet-based remote sensing image building change detection model and algorithm. Background Art

[0002] Buildings are the primary habitat for human life, production, and work, occupying vast tracts of land in urban and rural areas. Therefore, efficient and accurate building change detection plays a vital role in applications such as urban planning and agricultural land surveys. Remote sensing, with its macroscopic, large-scale, timely, periodic, and comprehensive nature, has become a crucial tool for monitoring the Earth's surface. Building change detection using remote sensing imagery aims to identify spatial and attribute changes in buildings within the same surface area using multi-temporal remote sensing imagery, and has become a crucial tool for detecting building changes.

[0003] Traditional remote sensing image change detection methods can be roughly divided into three categories: methods based on image algebraic operations, methods based on image transformation, and classification post-processing methods.

[0004] Methods based on image algebraic operations directly compare corresponding pixel values ​​in multi-temporal images to generate image difference maps. Based on this, thresholds are applied to classify pixels as either unchanged or changed. Methods based on image transformation transform image spectra, converting spectral combinations from the original feature space to a specific feature space to better distinguish changed pixels. Both of these methods are typical unsupervised approaches that rely heavily on empirically designed discriminative feature extraction algorithms and cannot achieve satisfactory results on high-spatial-resolution images.

[0005] Using supervised methods for change detection in remote sensing imagery can effectively improve its accuracy. The most commonly used method is classification post-processing. This method first uses supervised classification to classify objects in multi-temporal remote sensing imagery. It then compares the changes in each object category within the multi-period object classification map to generate an object change map. This circumvents the difficulty of detecting changes directly from raw imagery at different times. However, the accuracy of this method is heavily dependent on the accuracy of the classification results. The fine image details and complex texture features of high-spatial-resolution remote sensing imagery present new challenges for change detection.

[0006] With the rapid development of computer technology, humanity has entered the era of big data, and deep learning has demonstrated tremendous potential in various fields. Consequently, many deep learning-based change detection methods have been proposed, making image change detection more efficient and accurate. These neural networks include autoencoders (AEs), deep belief networks (DBNs), convolutional neural networks (CNNs), recurrent neural networks (RNNs), and generative adversarial networks (GANs). Convolutional neural networks, due to their unique advantages in image processing, have been widely used in the field of computer vision and have been successfully applied in tasks such as face recognition, object detection, semantic segmentation, and image classification. Building change detection in remote sensing images is a computer vision task, and therefore, using convolutional neural networks for building change detection in remote sensing images has important research significance and application value.

[0007] CenterNet is a popular convolutional neural network in the current field of computer vision, commonly used in tasks such as object detection and semantic segmentation. Its main feature is its ability to directly predict the center point and size of an object's bounding box. Compared to other object detection methods, this method does not require non-maximum suppression (NSS) during the prediction phase, greatly simplifying the network training and inference process. Furthermore, the CenterNet network has a very simple structure, a small number of model parameters, fast training speed, and is more user-friendly for platforms with lower computing power. However, the existing CenterNet network for building change detection in remote sensing imagery still has the following problems: 1. The C5 feature has an insufficient receptive field and cannot cover a sufficient number of large objects. 2. Using deconvolution for upsampling, because the convolution kernel size is not divisible by the stride, leads to a "checkerboard effect" in the feature map. Using ordinary bilinear interpolation for upsampling also causes misalignment.

[0008] In addition, in the task of building change detection, the number of pixels without building changes in most sample labels is much larger than the number of pixels with building changes, resulting in a serious class imbalance problem. Summary of the Invention

[0009] In response to the problems existing in existing remote sensing image building change detection technology, the present invention provides a remote sensing image building change detection model based on CenterNet, and a remote sensing image building change detection algorithm based on the model, which effectively improves the accuracy of building change detection in high spatial resolution remote sensing images.

[0010] The present invention protects a remote sensing image building change detection model based on CenterNet, including a feature extraction module, a building semantic segmentation module, and a change detection module;

[0011] The feature extraction module consists of two CenterNet networks with shared parameters. The Encoder module of the CenterNet network uses the ResNet18 network pre-trained based on the ImageNet dataset as the backbone network. The two phases of images are input into the feature extraction module respectively to obtain the corresponding feature maps.

[0012] The building semantic segmentation module includes a pyramid pooling module and a softmax classifier. After the feature map extracted from the two phases of images undergoes spatial pyramid pooling, the softmax classifier is used to predict pixel by pixel to form a semantic segmentation map of the changed area.

[0013] The change detection module includes a pyramid pooling module and a Sigmoid classifier, which calculates the absolute difference between the feature maps extracted from the two phases of images to obtain a difference feature map. After spatial pyramid pooling, the Sigmoid classifier is used to predict each pixel to form a building change map.

[0014] The loss function of the model is the joint cost loss function L = L ss1 +L ss2 +2*L wbce , where L ss1 , L ss2 are the semantic segmentation cost functions of the two phases of buildings, namely the cross entropy loss between the predicted value and the true value; L wbce The cost function for building change detection is the weighted binary cross entropy loss between the predicted value and the true value.

[0015] Furthermore, the weighted binary cross entropy loss L wbce =-(g i ·log(p i )·ω c +(1-g i )·log(1-p i )·ω u ), where g i Represents the true category label value of pixel i. When the building in pixel i changes, g i =1, otherwise g i =0;p i Indicates the probability of a building change in pixel i, 1-p i represents the probability that the building in pixel i has not changed; ω c 、ω uRepresent the loss weights of changed pixels and unchanged pixels, ω c =0.5 / f c ,ω u =0.5 / f u , f c 、f u are the proportions of building changed pixels and unchanged pixels in the sample, respectively.

[0016] Furthermore, the spatial pyramid contains four different pyramid scales, and a pooling operation is performed on the feature map to obtain feature maps of four scales. Then, "interpolation + 1×1 convolution" is used as an upsampling structure to restore the feature map to its initial size, and finally connected on the channel dimension.

[0017] Furthermore, the CenterNet network adopts an improved CenterNet, introduces a Dilate Encoder module, and uses "interpolation + 1×1 convolution" as an upsampling structure.

[0018] The present invention also protects a CenterNet-based remote sensing image building change detection algorithm, which uses the above remote sensing image building change detection model to detect building changes in two images, specifically comprising the following steps:

[0019] Step 1: Use the sample data set to train the model and update the network weights until the model verification accuracy no longer improves;

[0020] Step 2: Input the two images to be tested into the trained model to obtain the predicted building change model probability map;

[0021] Step 4: Segment according to the set threshold. If the probability value is greater than or equal to the threshold, it is determined to be a changed pixel and the corresponding pixel is reclassified as 255. If the probability value is less than the threshold, it is determined to be an unchanged pixel and the corresponding pixel is reclassified as 0.

[0022] Step 5: Splice the results according to the order of cropping and segmentation to obtain a building change map.

[0023] Furthermore, each group of samples in the sample dataset contains 5 images, namely image1 in period T1 and image2 in period T2, image1 and image2 are three-band true color composite images; building label maps label1 and label2 in the changed areas of image1 and image2, with pixel values ​​of 0, 1, and 2, corresponding to the unchanged area, non-buildings in the changed area, and buildings, respectively; two-period building change map label3, with pixel values ​​of 0 and 1, corresponding to the buildings in the area covered by the pixel having not changed and having changed, respectively.

[0024] Furthermore, the model training method includes the following steps:

[0025] Step a: Input a set of samples into the model to form a semantic segmentation map of the changed area and a building change map;

[0026] Step b: Calculate the cross entropy loss L between the semantic segmentation map of the changed area and label1 and label2 in the group of samples. ss1 , L ss2 ;

[0027] Step c, calculate the weighted binary cross entropy loss L by combining the building change map with label3 wbce ;

[0028] Step d, based on L ss1 , L ss2 and L wbce , calculate the joint cost loss function L, and then use the Adam optimizer to perform backpropagation and update the network weights.

[0029] The beneficial effects of the present invention are as follows: 1. A weighted binary cross entropy loss is used to construct a joint cost loss function for building change detection, which reduces the impact of category imbalance on model training and accelerates the model convergence speed and detection accuracy; 2. The four different pyramid scales of the spatial pyramid effectively retain global context information and improve the detection effect of buildings of different scales; 3. The Dilate Encoder module is introduced to improve the receptive field by stacking several dilated convolutions, thereby improving the detection ability of large buildings; 4. "Interpolation + 1×1 convolution" is used as the upsampling structure to replace the traditional deconvolution operation, which effectively weakens the checkerboard effect, solves the misalignment problem caused by ordinary bilinear interpolation upsampling, and improves the accuracy of feature extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a schematic diagram of the model structure;

[0031] Figure 2 This is a schematic diagram of the pyramid pooling module structure;

[0032] Figure 3 It is a schematic diagram of the improved CenterNet structure;

[0033] Figure 4 It is a schematic diagram of the offline building change detection process. DETAILED DESCRIPTION

[0034] The present invention will be described in further detail below with reference to the accompanying drawings and specific embodiments. The embodiments of the present invention are provided for purposes of illustration and description and are not intended to be exhaustive or to limit the invention to the disclosed forms. Many modifications and variations will be apparent to those skilled in the art. The embodiments are chosen and described to better illustrate the principles of the invention and its practical application, and to enable those skilled in the art to understand the invention and design various embodiments with various modifications suitable for specific applications.

[0035] Example 1

[0036] A remote sensing image building change detection model based on CenterNet, such as Figure 1 As shown in Figure 3, the model includes a feature extraction module, a change detection module, and a building semantic segmentation module.

[0037] The feature extraction module consists of two CenterNet networks with shared parameters. The Encoder module of the CenterNet network uses the ResNet18 network pre-trained based on the ImageNet dataset as the backbone network. The two phases of images are input into the feature extraction module respectively to obtain the corresponding feature maps.

[0038] The building semantic segmentation module includes a pyramid pooling module and a softmax classifier. It performs building semantic segmentation on the area where the ground object category changes in the feature map extracted from the two phases of images. After spatial pyramid pooling, the softmax classifier is used to predict pixel by pixel to form a building semantic segmentation map of the changed area.

[0039] The change detection module includes a pyramid pooling module and a Sigmoid classifier. It calculates the absolute difference between the feature maps extracted from the two phases of images to obtain a difference feature map. After spatial pyramid pooling, the Sigmoid classifier is used to predict pixel by pixel to form a building change map.

[0040] The loss function of the model is the joint cost loss function L = L ss1 +L ss2 +2*L wbce , where L ss1 , L ss2 are the semantic segmentation cost functions of the two phases of buildings, namely the cross entropy loss between the predicted value and the true value; L wbce The cost function for building change detection is the weighted binary cross entropy loss between the predicted value and the true value.

[0041] Binary cross-entropy loss is commonly used as the cost function for binary classification problems. However, in the building change detection task, the output change map contains only two categories: changed and unchanged. Therefore, it can be considered a binary classification problem. For change detection tasks, the ratio of changed and unchanged pixels in the label is generally very different. Therefore, it is necessary to weight the loss function based on the true class label to mitigate the impact of class imbalance.

[0042] The weighted binary cross entropy loss used in this example is defined as

[0043] L wbce =-(g i ·log(p i )·ω c +(1-g i )·log(1-p i )·ω u )

[0044] Among them, g i Represents the true category label value of pixel i. When the building in pixel i changes, g i =1, otherwise g i =0;p i Indicates the probability of a building change in pixel i, 1-p i represents the probability that the building in pixel i has not changed; ω c 、ω u Represent the loss weights of changed pixels and unchanged pixels, ω c =0.5 / i c ,ω u =0.5 / f u , f c 、f u are the proportion of pixels with building changes and pixels without building changes in the sample. Therefore, if the number of pixels with building changes in the label is less than the number of pixels with no building changes, the weight is greater than 1, making the contribution of the changed pixels to the loss value greater.

[0045] The spatial pyramid includes four different pyramid scales, such as Figure 2 As shown in the figure, the feature map is pooled to obtain feature maps of four scales. Then, "interpolation + 1×1 convolution" is used as the upsampling structure to restore the feature map to its initial size. Finally, it is connected in the channel dimension, which effectively retains the global context information and improves the detection effect of buildings of different scales.

[0046] In order to solve the technical problems existing in the application of the existing CenterNet network to remote sensing image building change detection, this embodiment adopts an improved CenterNet, introduces the Dilate Encoder module, and uses "interpolation + 1×1 convolution" as the upsampling structure. Figure 3 The Dilate Encoder module is introduced to increase the receptive field by stacking several dilated convolutions, improving the detection ability of large buildings. The "interpolation + 1×1 convolution" upsampling structure replaces the traditional deconvolution operation, effectively reducing the checkerboard effect and improving the accuracy of feature extraction.

[0047] Example 2

[0048] A CenterNet-based remote sensing image building change detection algorithm uses the remote sensing image building change detection model described in Example 1 to detect building changes in two phases of images.

[0049] 1. Sample Set Construction and Division

[0050] The sample set consists of a public dataset and a self-produced dataset. The public dataset is the SECOND semantic change detection dataset, with images of 512×512 pixels and pixel-level annotation. To improve model training, this example further expands the dataset and generates samples using two phases of high-resolution remote sensing imagery of the same region, using ArcGIS. The two phases mentioned herein represent the same region at different time periods.

[0051] The final sample dataset contains a total of 6813 groups of samples, each group of samples contains 5 images, namely image1 in period T1 and image2 in period T2, with a size of 512×512×3, which are three-band true color composite images; building label maps label1 and label2 in the changed areas of image1 and image2, with a size of 512×512, and pixel values ​​of 0, 1, and 2, corresponding to the unchanged area, non-buildings in the changed area, and buildings, respectively; label3, a building change map of the two periods, with a size of 512×512, and pixel values ​​of 0 and 1, corresponding to the buildings in the area covered by the pixels having not changed and having changed, respectively.

[0052] The sample data set is divided into 9 ∶ 0.5 ∶ The set is divided into training, validation and test sets with a ratio of 0.5.

[0053] 2. The model training process for a group of samples is as follows:

[0054] 1. Read image1 and image2, input them into the feature extraction module for feature extraction, and output two feature maps.

[0055] 2. In the building semantic segmentation module, the feature map extracted from the two phases of images is interpolated after spatial pyramid pooling, and the feature map size is restored to 512×512. Then, the softmax classifier is used to predict each pixel, and the cross entropy loss L is calculated with label1 and label2 respectively. ss1 , L ss2 .

[0056] 3. In the building change detection module, the absolute difference of the feature maps extracted from the two phases of images is calculated to obtain the difference feature map. After spatial pyramid pooling, interpolation is performed to restore the feature map size to 512×512. Then, pixel-by-pixel prediction is performed using the sigmoid classifier, and the weighted binary cross entropy loss L is calculated with label3. wbce .

[0057] 4. Based on L ss1 , L ss2 and L wbce , calculate the joint cost loss function L, and then use the Adam optimizer to perform backpropagation and update the network weights.

[0058] 3. Offline Building Change Detection

[0059] 1. After preprocessing the two-period remote sensing images to be tested, a true color composite three-band slice image is obtained with an image size of 512×512×3.

[0060] 2. Input the two-phase images into the model to obtain the predicted building change probability map. If the segmentation threshold is set to 0.5, the pixels with probability values ​​greater than or equal to 0.5 are pixels where building changes have occurred, and the corresponding pixel values ​​are reclassified as 255. The pixels with probability values ​​less than 0.5 are pixels where no building changes have occurred, and the pixel values ​​are reclassified as 0.

[0061] 3. According to the order of cutting and segmenting, the results are spliced ​​to obtain the building change map after detection, such as Figure 4 shown.

[0062] Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field and related fields without making any creative work should fall within the scope of protection of the present invention.

Claims

1. A CenterNet-based remote sensing image building change detection algorithm, characterized by: The building changes in the two images were detected using a remote sensing image building change detection model, which includes a feature extraction module, a building semantic segmentation module, and a change detection module. The feature extraction module consists of two CenterNet networks with shared parameters. The Encoder module of the CenterNet network uses the ResNet18 network pre-trained based on the ImageNet dataset as the backbone network. The two phases of images are input into the feature extraction module respectively to obtain the corresponding feature maps. The building semantic segmentation module includes a pyramid pooling module and a softmax classifier. After the feature map extracted from the two phases of images is subjected to spatial pyramid pooling, the softmax classifier is used to predict pixel by pixel to form a semantic segmentation map of the building in the changed area. The change detection module includes a pyramid pooling module and a Sigmoid classifier, which calculates the absolute difference between the feature maps extracted from the two phases of images to obtain a difference feature map. After spatial pyramid pooling, the Sigmoid classifier is used to predict each pixel to form a building change map. The loss function of the model is the joint cost loss function L = L ss1 +L ss2 +2*L wbce , where L ss1 , L ss2 are the semantic segmentation cost functions of the two phases of buildings, namely the cross entropy loss between the predicted value and the true value; L wbce The cost function for building change detection is the weighted binary cross entropy loss between the predicted value and the true value; The detection specifically includes the following steps: Step 1: Use the sample data set to train the model and update the network weights until the model verification accuracy no longer improves; Each sample in the sample dataset contains 5 images, namely image1 in period T1 and image2 in period T2. Image1 and image2 are three-band true color composite images; building label maps label1 and label2 in the changed areas of image1 and image2, with pixel values ​​of 0, 1, and 2, corresponding to unchanged areas, non-buildings in changed areas, and buildings, respectively; label3, a building change map for the two periods, with pixel values ​​of 0 and 1, corresponding to unchanged and changed buildings in the areas covered by the pixels, respectively; The model training method includes the following steps: Step a: Input a set of samples into the model to form a semantic segmentation map of the changed area and a building change map; Step b: Calculate the cross entropy loss L between the semantic segmentation map of the changed area and label1 and label2 in the group of samples. ss1 , L ss2 ; Step c, calculate the weighted binary cross entropy loss L by combining the building change map with label3 wbce ; Step d, based on L ss1 , L ss2 and L wbce , calculate the joint cost loss function L, and then use the Adam optimizer to perform backpropagation and update the network weights; Step 2: Input the two images to be tested into the trained model to obtain the predicted building change model probability map; Step 4: Segment according to the set threshold. If the probability value is greater than or equal to the threshold, it is determined to be a changed pixel and the corresponding pixel is reclassified as 255. If the probability value is less than the threshold, it is determined to be an unchanged pixel and the corresponding pixel is reclassified as 0. Step 5: Splice the results according to the order of cropping and segmentation to obtain a building change map.

2. The CenterNet-based remote sensing image building change detection algorithm according to claim 1, characterized in that: Weighted binary cross entropy loss L wbce =-(g i ·log(p i )·ω c +(1-g i )·log(1-p i )·ω u ), where g i Represents the true category label value of pixel i. When the building in pixel i changes, g i =1, otherwise g i =0;p i Indicates the probability of a building change in pixel i, 1-p i represents the probability that the building in pixel i has not changed; ω c 、ω u Represent the loss weights of changed pixels and unchanged pixels, ω c =0.5 / f c ,ω u =0.5 / f u , f c 、f u are the proportions of building changed pixels and unchanged pixels in the sample, respectively.

3. The CenterNet-based remote sensing image building change detection algorithm according to claim 1, characterized in that: The spatial pyramid contains four different pyramid scales. Pooling operation is performed on the feature map to obtain feature maps of four scales. Then, "interpolation + 1×1 convolution" is used as the upsampling structure to restore the feature map to its initial size, and finally concatenate it in the channel dimension.

4. The CenterNet-based remote sensing image building change detection algorithm according to claim 3, characterized in that: The CenterNet network adopts the improved CenterNet, introduces the Dilate Encoder module, and uses "interpolation + 1×1 convolution" as the upsampling structure.

Images