Urbanization semantic change detection method based on high-resolution satellite image

Abstract

The embodiment of the invention discloses an urbanization semantic change detection method based on a high-resolution satellite image, and the method comprises the steps: collecting the high-resolution satellite image, carrying out the preprocessing, data annotation and data enhancement of the image, and constructing an urbanization semantic change data set; inputting the urbanization semantic change data set after data enhancement into an urbanization semantic change detection model (CT-Net), and obtaining urbanization change trends of different time sequences; and based on the urbanization change result, counting the urbanization expansion and decline percentage and area of the target area. According to the calculation scheme provided by the embodiment of the invention, fine-grained boundary detection and small-cavity regional detection are realized in the high-resolution image according to a visual urbanization sample, namely the impervious surface, and statistical calculation of percentage and area of city expansion and recession of the target area is realized, so that high-precision and automatic urbanization change extraction is realized.

Description

Technical Field

[0001] The present invention relates to the field of remote sensing, and in particular to a method for detecting urbanization semantic changes based on high-resolution satellite images, which is mainly applicable to analyzing urbanization change trends and realizing visualization. Background Art

[0002] With the rapid development of society and economy, urbanization expansion is an inevitable result of urban development, but irrational and disorderly expansion will have a great negative impact on land resources and natural environment. A comprehensive and detailed understanding of the changes and trends of urban urbanization is an important guarantee for the sustainable development of cities. By detecting the changing trend and speed of impervious surfaces in urban fringe areas, the overall situation of urban development can be basically reflected. Continuous monitoring can objectively evaluate the degree of urbanization expansion and decline in the target area, which plays an important role in urban planning and sustainable development management of cities.

[0003] Change detection based on satellite images is a technology that uses one or more sets of images taken at different times at the same geographical location, combined with the texture and shape information of the objects, to identify the addition and decline of objects. With the advancement of remote sensing technology, it is now possible to continuously receive multi-period image data through a variety of remote sensing platforms (such as WorldView-3, GeoEye-1, QuickBird, GaoFen-2), which greatly promotes the development of change detection technology.

[0004] At present, the methods for change detection can be roughly divided into two categories: based on image feature extraction and based on deep learning. Among them, the change detection methods based on image feature extraction are mainly aimed at multispectral images, extracting rich spectral, texture and structural features in the images, such as principal component analysis, change vector and multivariate change detection. Such methods are simple to operate and widely used in the field of change detection. However, because their feature extraction usually relies on expert manual design, they cannot effectively capture key information when facing different lighting, weather conditions and seasonal changes, and their performance and automation are low. The change detection method based on deep learning has a higher degree of automation and large-scale data processing capabilities. It can bring a larger receptive field and context information extraction capability to the change extraction network by combining attention mechanisms (such as spatial attention, channel attention, global attention, etc.) and multi-scale modules in convolutional neural networks. In recent years, related studies have begun to introduce Transformer into convolutional neural networks and have achieved good detection results, and have made great improvements in fine-grained boundary delineation and feature extraction. For example, Chen et al. first introduced Transformer to extract change features, compressed the feature map of each time series into tokens through spatial attention, modeled the spatial context, and obtained global features. For example, Chang et al. improved SegFormer by using a twin network structure built entirely by Transformer to concatenate multi-scale features and obtain multi-scale global information. At the same time, with the introduction of ConvNext, it was proved that CNN has the performance potential comparable to Transformer. Both have their own advantages and disadvantages in terms of receptive field size and context modeling.

[0005] In summary, as the resolution of satellite images continues to improve, the images contain clearer geometric structures and more complex texture information, which will cause traditional extraction methods to produce too many pseudo-changes and greatly increase the difficulty of extraction. How to construct a model with high-precision extraction capabilities, high degree of automation, and large-scale processing of image data through deep learning is an urgent task to be achieved. Summary of the invention

[0006] The purpose of the present invention is to provide a method for detecting urbanization semantic changes based on high-resolution satellite images, so as to achieve accurate detection and visualization of urbanization change trends.

[0007] To achieve the above purpose, the present invention provides the following steps: A. Obtain satellite images of the same area at different time series through the Google Earth platform, pre-process the images, and obtain calibrated image data; B. Perform data annotation, data partitioning and data enhancement on the calibrated images to generate an urbanization semantic change data set; C. Input the urbanization semantic change data set into an urbanization semantic change detection model (CT-Net) to obtain the urbanization change trend of each image; D. Based on the detection results, the percentage and area of ​​urbanization expansion, decline, and unchanged in the target area are counted.

[0008] Wherein, the step A specifically includes:

[0009] The acquired previous time series image is used as the reference, and the obvious point areas in the dual time series image are used as control points (such as intersections and building corners) to align the vector data of the dual time series;

[0010] Then the target area in the image is cropped to obtain a group of images at the same geographical location but different time series.

[0011] The step B specifically comprises:

[0012] The preprocessed image group is labeled, and then the detection area is divided into data, and the image is cut into 512×512 size;

[0013] Rotate the image 90° or 270°, or flip the image to perform the first step of image data enhancement;

[0014] The image contrast is adjusted to 60% or 120% of the original image, and the second step of image data enhancement is performed.

[0015] The step C specifically comprises:

[0016] The CT-Net encoder consists of a shallow network and a deep network;

[0017] The shallow network consists of a convolutional neural network, which focuses on local subtle changes in the ground objects;

[0018] The deep network consists of convolutional neural networks and Transformers, which focus on regional changes in objects and expand the receptive field to capture broader contextual information;

[0019] The dual-time series feature fusion structure of CT-Net is used to realize the fusion of nonlinear relationships in dual time series;

[0020] The CT-Net decoder consists of a multi-scale hybrid module and MLP, which performs multi-scale fusion on the dual temporal feature fusion results to unify the dimensions of each channel;

[0021] The CT-Net network is used to detect the image group and obtain the trend of urbanization changes.

[0022] The step D specifically comprises:

[0023] The detection results of the CT-Net network were analyzed, and the number of urban expansion, urban decline, and unchanged pixels in the detection results were counted. Combined with the resolution of the high-resolution images, the total expansion, decline, and unchanged area and percentage were further calculated. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a flow chart of the urbanization semantic change detection of the present invention;

[0025] Figure 2 An example of an urbanization semantic change detection dataset provided by an embodiment of the present invention;

[0026] Figure 3 A schematic diagram of the overall structure of the deep learning network CT-Net provided in an embodiment of the present invention;

[0027] Figure 4 A schematic diagram of a DCA module in a deep learning network CT-Net provided in an embodiment of the present invention;

[0028] Figure 5 A schematic diagram of a CTF module in a deep learning network CT-Net provided in an embodiment of the present invention;

[0029] Figure 6 A schematic diagram of a Depthwise ASPP module in a deep learning network CT-Net provided in an embodiment of the present invention; DETAILED DESCRIPTION

[0030] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0031] See also Figure 1 , which is an operation flow chart of the urbanization semantic change detection method based on high-resolution satellite images of the present invention.

[0032] Step S1, obtain high-resolution satellite images on the Google Earth platform, and obtain the target area image through image registration and image cropping. Specifically:

[0033] In this example, Urumqi, Xinjiang Uygur Autonomous Region, China, was selected as the example area during the experiment. The example area was obtained through the Google Earth platform, including high-resolution satellite images of Urumqi in 2017 and 2021, with an average resolution of 0.65m. After that, the intersections and significant ground feature marking control points in the images of each year were aligned with the two time series images, and then the example area was cropped to obtain the processed image.

[0034] Step S2, data division and data enhancement are performed on the processed image. Specifically:

[0035] Step S21, labeling the processed data, taking the impervious surface as a sample, where "background changes to impervious surface", "impervious surface changes to background" and "unchanged" correspond to "urbanization expansion", "urbanization decline" and "unchanged" respectively;

[0036] Step S22, dividing the labeled images into data, cutting the images into 512×512 sizes, and obtaining 1836 groups of dual-time series images in total;

[0037] Step S23, flipping or rotating the image, wherein the image is rotated by 90° or 270° to perform data enhancement to expand sample diversity;

[0038] Step S24, the contrast of the image is changed to 60% or 270% of the original image, and finally a total of 7344 sets of urbanization change detection image groups are obtained (please also refer to Figure 2 );

[0039] Step S25, among the 7344 sets of urbanization change detection image groups constructed, 70% of the data are used as training sets, 10% of the data are used as validation sets, and 20% of the data are used as test sets.

[0040] Step S3, using the urbanization change detection model CT-Net (see also Figure 3 ), end-to-end pixel-level segmentation of urbanization semantic changes in satellite images, including:

[0041] Step S31, the left half of the CT-Net architecture is the encoder, which is divided into a shallow part and a deep part, focusing on local subtle changes and regional changes respectively, including:

[0042] Step S311, the encoder is composed of 4 stages, stages 1 and 2 are shallow parts, and stages 3 and 4 are deep parts;

[0043] Step S312: The shallow part of the encoder is composed of a DCA module (see also Figure 4), the DCA module is a convolutional neural network structure, which consists of a residual-like structure and a feature extraction structure with channel attention. The execution process is:

[0044]

[0045] H1(x)=C 1×1 (MP(x))

[0046]

[0047] M1(x)=σ(BN(D 3×3 (BN(x))))

[0048] M2(x)=C 1×1 (σ(C 1×1 (GP(x))))

[0049] M3(x)=C 1×1 (Φ(C 1×1 (AP(x))))

[0050] Among them, C 1×1 , C 3×3 and D 3×3 represents convolution kernels of sizes 1×1, 3×3 and 3×3 respectively, BN represents batch normalization layer, σ represents linear correction unit function GELU, GP and MP represent global average pooling and maximum pooling respectively.

[0051] Step S313, the deep network of the encoder is composed of a Depthwise ASPP module (see also Figure 5 ) and CTF modules (see also Figure 6 )constitute;

[0052] Step S314, at the beginning of stage 3 and stage 4, the feature map is input into the Depthwise ASPP. By introducing the Depthwise convolution and ASPP structure, the acquired feature map establishes a preliminary connection between high-dimensional local features. The execution process is as follows:

[0053] Depthwise ASPP=Φ(BN(C 1×1 (concat(A1,A2))))

[0054] A1=UP(Φ(BN(C 1×1 (x))))

[0055] A2=concat(BN(m1),BN(m2),BN(m3),BN(m4))

[0056]

[0057] Among them, Φ represents the linear correction unit function ReLU, C 1×1 represents a 1×1 convolution kernel, BN represents a batch normalization layer, UP represents upsampling, and D 3×3 Depthwise convolution kernel with size 3×3 and interval n. The main structure of Depthwise ASPP consists of 5 parallel branches, including 4 dilated convolution branches with intervals of [1, 6, 12, 18] and 1 1×1 convolution kernel. Concat is feature concatenation.

[0058] Step S315, the CTF module is constructed by fusion of convolutional neural network and Transformer, and the above DCA module is used as the input structure of the CTF module to extract local changes in the deep network, and then the acquired feature map is linearly projected, and the projection features are input into the Transformer, and divided through a multi-head mechanism, so that the attention is modeled by calculating the weighted average in each head to model the global dependency. The multi-head attention execution process is:

[0059] MSA=concat(h1,h2,...,h N )·W

[0060]

[0061] Where: concat(·) represents the output result of the concatenated multiple heads, softmax(·) represents the softmax function, K T is the transpose of K, d is the normalization factor, Represents a linear projection matrix.

[0062] Step S32, the right part of CT-Net is the decoder part, which is composed of a multi-scale hybrid module and MLP, specifically including:

[0063] Step S321, the dual time series features obtained from the encoder are fused through a multi-scale hybrid module, and the execution process is:

[0064] F s T1-T2 =σ(C 3×3 (BN(σ(C 3×3 (concat(F i T1 ,F i T2 ))))))

[0065] Among them, C 3×3represents a 3×3 convolution kernel, σ represents a linear rectification unit function GELU, and BN represents a batch normalization layer.

[0066] Step S322: The main part of the decoder is composed of MLP, and the execution process is:

[0067]

[0068] Where: MLP stands for multi-layer perceptron, and UP stands for upsampling.

[0069] Step S33, using the encoder and decoder structure of the CT-Net network to segment the dual-time series images, the segmentation results obtained include:

[0070] True positives (TP): the number of instances that are correctly classified as positive examples, that is, the number of instances (number of samples) that are actually positive examples and are classified as positive examples by the network;

[0071] False positives (FP): the number of instances that are mistakenly classified as positive examples, that is, the number of instances that are actually negative examples but are classified as positive examples by the network;

[0072] False negatives (FN): The number of instances that are mistakenly classified as negative examples, that is, the number of instances that are actually positive examples but are classified as negative examples by the network;

[0073] True negatives (TN): the number of instances that are correctly classified as negative examples, that is, the number of instances that are actually negative examples and are classified as negative examples by the network;

[0074] The TP, FP, FN, and TN obtained through urbanization change detection are further evaluated through OA, Precision, Recall, F1, and MIoU to evaluate the detection ability of the model. The formula execution process is:

[0075]

[0076] OA is a measure of the proportion of samples that are correctly predicted by the model as a whole, indicating the proportion of correctly classified samples to the total number of samples;

[0077] Precision indicates the proportion of samples that are actually positive among all samples predicted to be positive.

[0078] Recall represents the proportion of samples that are correctly predicted to be positive among all samples that are actually positive.

[0079] F1 is the harmonic mean of Precision and Recall, which is used to comprehensively weigh Precision and Recall to ensure that the model has good performance in both indicators;

[0080] MIoU indicates the degree of overlap between the predicted result and the actual label. The value range of MIoU is between 0 and 1. The larger the value, the higher the degree of overlap between the predicted result and the actual label.

[0081] Step S4, based on the urbanization change detection results obtained in the previous steps, by counting the change percentages of the three labels of urbanization expansion, urbanization decline, and unchanged, and combining the resolution of the image, calculate the urbanization expansion area, urbanization decline area, and urbanization unchanged area.

Claims

1. A method for detecting urbanization semantic changes based on high-resolution satellite images, characterized in that: include: A. Obtain high-resolution images of the same area at different time series on the Google Earth platform, perform image registration and image cropping on the images, and obtain calibrated data; B. Annotate, partition and enhance the calibrated data to generate a dataset of urbanization semantic changes; C. Input the urbanization semantic change dataset into the urbanization semantic change detection model (CT-Net) to obtain the urbanization change results of each image group; D. Based on the detection results of each set of images, calculate the percentage and area of ​​urban expansion, decline, and unchanged in the target area.

2. The method according to claim 1, characterized in that The step A specifically comprises: For each image group, the previous time-series image is used as the reference, and obvious point areas (such as intersections and building corners) in the dual-time-series images are selected as control points to align the dual-time-series images; The target area in the complete image is cropped to obtain image groups of the same geographical location but different time series.

3. The method according to claim 1, characterized in that The step B comprises: The acquired image groups were labeled, and the impervious surface was used as a sample to label "background becomes impervious surface", "impervious surface becomes background" and "unchanged", which correspond to "urbanization expansion", "urbanization decline" and "unchanged" respectively; Divide the data into 512×512 sizes; Rotate the image group by 90° or 270° at the same time, or flip the image to perform the first step of image data enhancement; The image contrast is adjusted to 60% or 120% of the original image, and the second step of image data enhancement is performed.

4. The method according to claim 1, characterized in that The step C comprises: The encoder of CT-Net consists of a shallow network and a deep network; The shallow network of CT-Net is composed of a convolutional network, which includes channel attention and residual structure, and mainly focuses on subtle changes in the ground objects; The deep network of CT-Net is composed of a fusion of convolutional networks and Transformers. Transformers focus on regional changes and expand the receptive field, while convolutional neural networks focus on subtle changes in the boundaries of regional changes. The dual-time series feature fusion structure of CT-Net is used to splice the dual-time series features extracted by the encoder to achieve nonlinear data fusion; The decoder of CT-Net is composed of a multi-scale hybrid module and MLP, which fuses the dual-time series feature fusion results to unify the dimensions of each channel; the CT-Net network is used to detect dual-time series satellite images to obtain the changing trend of urbanization.

5. The method according to claim 1, characterized in that The step D comprises: The detection results of the CT-Net network were analyzed, and the number of urban expansion, urban decline, and unchanged pixels in the detection results were calculated. Combined with the resolution of the high-resolution images, the area and percentage of urban expansion and decline were calculated.

Images