Building Change Detection Method Based on Orthophoto and Oblique Photogrammetry Data

Abstract

The present invention is applicable to the field of image processing technology, and provides a building change detection method based on orthophotos and oblique photography data, the method comprising: for the same area, obtaining two orthophotos at different times, obtaining raster patches of building outline changes through building extraction and building outline change detection, and further converting them into vector patches; step S2, for the area, obtaining oblique photography data of the period corresponding to the two orthophotos, obtaining building height change vector patches through building height change detection; step S3, fusing the building outline change vector patches with the height change vector patches to obtain the overall building change patches. The present invention automatically extracts building patches from remote sensing image data through a deep learning algorithm, and can achieve high-precision automatic identification of buildings. Moreover, through building outline change detection and building height change detection, it can determine whether the outline and height of the building have changed, and can quickly determine the current suspected illegal buildings, which can significantly improve the efficiency of illegal building identification.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to a building change detection method based on orthophoto and oblique photography data. Background Art

[0002] In recent years, the national policy level has successively issued a number of guiding opinions on the investigation and punishment of illegal buildings in urban management. The work of investigating and punishing illegal buildings has sprung up across the country. It is possible to judge whether a building is an illegal building by observing the changes in the building.

[0003] Current building image detection solutions typically involve regularly acquiring high-resolution satellite imagery and employing traditional methods to extract handcraft features to identify illegal structures. However, this method is complex and inaccurate, requiring manual confirmation after identifying illegal structures. This significantly increases the difficulty and impacts efficiency. Summary of the Invention

[0004] In view of the above problems, the purpose of the present invention is to provide a building change detection method based on orthophoto and oblique photography data, aiming to solve the technical problem of low accuracy in extracting man-made features.

[0005] The present invention adopts the following technical solutions:

[0006] The building change detection method of orthophoto and oblique photography data comprises the following steps:

[0007] Step S1: For the same area, two phases of orthophotos are obtained, and raster patches of building outline changes are obtained through building extraction and building outline change detection, which are further converted into vector patches;

[0008] Step S2: Obtain oblique photography data corresponding to the two periods of orthophotos for the area, and obtain building height change vector maps by detecting building height changes;

[0009] Step S3: Fusing the building outline change vector image patch and the height change vector image patch to obtain the overall building change image patch.

[0010] Furthermore, the specific process of step S1 is as follows:

[0011] S11. Train a deep learning-based building recognition model by collecting data;

[0012] S12, obtaining two orthophotos of the same area, and using the building recognition model to predict the building patterns in the orthophotos to obtain building patches;

[0013] S13. Use Tensor Polygonization to generate a simplified building outline vector from the building patch, and obtain a building outline change vector patch based on the changes in the building outline vectors in two phases.

[0014] Furthermore, in step S11, the data collection includes collecting the required high-resolution remote sensing data, recording the shooting time and height of the high-resolution remote sensing data, preprocessing the data, and resampling the data to the required resolution.

[0015] Furthermore, in step S11, the building recognition model includes an encoding network, a decoding network and a pixel-by-pixel classifier.

[0016] Furthermore, in step S13, the process of generating a simplified building outline vector is as follows:

[0017] Generate building heat map through building recognition model and obtain building vector;

[0018] According to the principle of building shape characteristics, the building vector is smoothed to eliminate the jagged edges in the generated vector;

[0019] Identify building corners and mark corner locations based on the directions generated in the building recognition model;

[0020] Delete the vectors between corners and optimize the position of generated building outline vectors.

[0021] Furthermore, the specific process of step S2 is as follows:

[0022] S21, obtaining oblique photography data corresponding to the two periods of orthophotos, merging the triangulated networks of the oblique photography data to form a single large-scale oblique photography data;

[0023] S22, using the building spot outline data to be superimposed on the merged oblique photography data, where the oblique photography data carries high-level information of all pixels, obtaining the highest high-level information within the oblique photography data range as a high-level value, and writing the high-level value as an attribute field into the current building spot outline data;

[0024] S23. After obtaining the building outline data with high-rise values in two periods, matching is performed according to the range coordinates, and then the high-rise values are compared to obtain the height change vector map of the building whose high-rise values have changed.

[0025] The beneficial effects of the present invention are: the present invention automatically extracts buildings from massive data through a deep learning algorithm, and can realize high-precision automatic identification of buildings. Moreover, through building outline change detection and building height change detection, it can be determined whether the outline and height of the building have changed. As long as the outline or height of the building has changed, the current building can be determined to be an illegal building. This method is accurate in judgment, simple to implement, and can significantly improve the efficiency of illegal building identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 is a flow chart of a method for detecting building changes using orthophoto and oblique photography data provided by an embodiment of the present invention;

[0027] Figure 2 is a schematic diagram of a building recognition model provided by an embodiment of the present invention;

[0028] Figure 3 This is a schematic diagram of directions for adding to the building recognition model provided by an embodiment of the present invention;

[0029] Figure 4 This is a schematic diagram of the Tensor Polygonization vectorization process provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0031] In order to illustrate the technical solution of the present invention, specific embodiments are provided below.

[0032] Figure 1 The flow of the building change detection method based on orthophoto and oblique photography data provided by an embodiment of the present invention is shown. For ease of explanation, only the part related to the embodiment of the present invention is shown.

[0033] like Figure 1 As shown, the building change detection method using orthophoto and oblique photography data provided in this embodiment includes the following steps:

[0034] Step S1: For the same area, two phases of orthophotos are obtained, and raster patches of building outline changes are obtained through building extraction and building outline change detection, and further converted into vector patches.

[0035] Orthophotos are remote sensing images with orthographic projection properties. Original remote sensing images are affected by changes in the sensor's internal state, external conditions, and surface conditions during imaging, resulting in varying degrees of distortion and loss of stability. Geometric processing of remote sensing images not only extracts spatial information but also resamples the image's grayscale according to the correct geometric relationships, creating a new orthophoto.

[0036] For buildings, the original images taken by remote sensing are geometrically processed to obtain orthophotos of the buildings. The specific processing process is not the focus of this embodiment and will not be described in detail here.

[0037] This step obtains building outline data through contour change detection, and finally obtains building outline change vector patches through the orthophoto building outline data of two periods. The specific process is as follows:

[0038] S11. Train a deep learning-based building recognition model by collecting data.

[0039] Data collection includes collecting the required high-resolution remote sensing data and recording its shooting time and height. The data is pre-processed, resampled to the required resolution and imported; then, based on the existing building and road data, a building recognition model based on encoder-decoder deep learning is trained. This model is used for building extraction. Figure 2 As shown in the figure, the building recognition model mainly consists of an encoder network, a decoder network and a pixel-wise classification layer.

[0040] The encoding network converts high-dimensional vectors into low-dimensional vectors, enabling low-dimensional extraction of high-dimensional features. While the encoding network captures more translation-invariant features through multiple max pooling operations, it also loses more feature map boundary information, which is important for segmentation. Therefore, the max pooling index is recorded during the pooling process to preserve the location of the maximum eigenvalue. This max pooling index is then used to upsample the input feature map, preserving boundary information. The max pooling index further optimizes the accuracy of boundary contour descriptions, reduces the number of parameters, and allows for end-to-end training, further improving recognition accuracy.

[0041] The decoding network uses the maximum pooling index information of the corresponding feature layer saved during encoder downsampling to map low-resolution feature maps to high-spatial-resolution feature maps, achieving reconstruction from low-dimensional vectors to high-dimensional vectors. Reusing the maximum pooling index during decoding has several advantages: it optimizes boundary contour description; it reduces the number of parameters and allows end-to-end training; and the upsampling method can be applied to any encoder-decoder network.

[0042] The encoding network and the decoding network are connected via a band attention connection, which enables the model to read the original high-dimensional data again during decoding, ensuring that the geometric features remain unchanged.

[0043] In the last layer, the decoder outputs a high-dimensional feature representation vector as the input of the trainable classifier. The encoding network can use the classic Resnet, Inception or Efficientnet.

[0044] In the specific implementation, this building recognition model is based on PyTorch and is trained on a server with three GeForce RTX2080Ti GPUs. The training batch size is 64 (batch_size), the learning rate is 1e-04, AdamW is used to optimize the parameters, and the cross entropy loss is used as the loss model to constrain the model training. The training process is as follows:

[0045] 1. The training data size is 512*512, and it is obtained from the original image using a sliding window method. The number of iteration rows is 256, ensuring that most buildings exist independently in a single image. 60% of the total data is selected as training data and 40% as validation data.

[0046] 2. Random image augmentation is used during training to improve model generalization and avoid overfitting. Images are randomly augmented using methods such as inversion, scaling, random brightness, and contrast.

[0047] 3. Model training uses a fusion loss function, including cross entropy and the Dice coefficient. Cross entropy is used to ensure the model's accurate identification of building types, while the Dice coefficient is used to constrain building outlines.

[0048] 4. The model is trained for 400 cycles. The initial learning rate is 1e-04 and is gradually reduced to 1e-07 after 10 cycles. The model with the highest accuracy on the test data is selected as the final model.

[0049] S12. Obtain two orthophotos of the same area, and use a building recognition model to predict building patterns in the orthophotos to obtain building patches.

[0050] To predict whether a building's outline has changed, we need to compare the orthophotos of the building from two different periods. This is then directly input into the building recognition model trained in the previous step to generate the building patch.

[0051] S13. Use Tensor Polygonization to generate a simplified building outline vector from the building patch, and obtain a building outline change vector patch based on the changes in the building outline vectors in two phases.

[0052] Deep learning image segmentation usually outputs the segmentation results in raster format, while the application of geographic information systems often requires vector output. In order to optimize the process of raster image vectorization, this step uses Tensor Polygonization to perform vectorization processing. Figure 3 As shown in Figure 2, we first added a direction output to the building recognition model to make it consistent with the ground truth contour. This is used to predict the direction of each current target and thus extract a smooth vector effect. The vectorization process is referenced in

[15] . Figure 4 As shown, the process is as follows:

[0053] 1. Generate a building heat map through the building recognition model and obtain the building vector;

[0054] 2. Smooth the building vectors based on the building shape characteristics to eliminate jagged edges in the generated vectors;

[0055] 3. Identify building corners and mark corner locations based on the directions generated in the building recognition model;

[0056] 4. Delete the vectors between corners and optimize the position of the generated building outline vector.

[0057] The simplified vector generation method (Tensor Polygonization) in this step, based on the characteristic that building outlines are mostly rectangular or equiangular polygons, proposes a vectorization method based on building direction, thereby significantly optimizing the accuracy and data volume of generating building vectors from image heat maps.

[0058] Step S2: Obtain oblique photography data corresponding to the two periods of orthophotos for the area, and obtain building height change vector maps by detecting building height changes.

[0059] S21. Obtain oblique photography data corresponding to the two phases of orthophotos, and merge the triangulated networks of the oblique photography data to form a single large-scale oblique photography data.

[0060] Oblique photography data is relative to orthophotos. To determine if a building is illegal, this step requires obtaining oblique photography data from the same period as two orthophotos of the building. The orthophotos for periods A and B are Orthophoto A and Orthophoto B, respectively, and the oblique photography data A and B are also obtained for the same period.

[0061] Since oblique photography data is composed of multiple separate OSGB data, when a building pattern spans multiple OSGB data, there is a loss of efficiency in data reading and retrieval. Therefore, this embodiment merges the various OSGB data into a large OSGB data to improve data reading efficiency during retrieval. Oblique photography OSGB data is generally a continuous triangular mesh. Merging multiple OSGB data requires merging the vertices and indices in each data to ensure that the merged OSGB data has only one triangular mesh and one texture.

[0062] This step reassembles the fragmented original OSGB oblique photography data into a large-scale OSGB data. When reading the data at a high level, it reduces the number of original data traversals and reduces data throughput.

[0063] S22. Use the building pattern outline data to superimpose on the merged oblique photography data, that is, the orthophoto A is superimposed on the oblique photography data A, and the orthophoto B is superimposed on the oblique photography data B. The oblique photography data has high-level information of all pixels. The highest high-level information within the range of the oblique photography data is obtained as the high-level value. The high-level value is used as an attribute field and written into the current building pattern outline data. The high-level information of the building can be obtained through high-level extraction.

[0064] S23. After obtaining the building outline data with high-rise values in two periods, matching is performed according to the range coordinates, and then the high-rise values are compared to obtain the height change vector map of the building whose high-rise values have changed.

[0065] In the process of extracting the high-level image spots, since the original oblique photography data are all stored in fragments, the preprocessing of the original oblique photography data is very important. In this step, the efficiency of reading the high-level information of the oblique photography data is further improved by reorganizing the oblique photography data.

[0066] Step S3: Fusing the building outline change vector image patch and the height change vector image patch to obtain the overall building change image patch.

[0067] The overall building change map represents the changes in the building's outline and height. Through the building change map, as long as any change is found in either the outline or the height, the building can be determined to be an illegal construction. In this way, it is possible to quickly and accurately determine whether a building is an illegal construction. The whole process is simple to implement. It only requires training the building recognition model and obtaining the orthophoto and oblique photography data of the same period. Through data processing and detection judgment, it is possible to quickly identify whether the building's outline and height have changed. This method can significantly improve recognition accuracy and efficiency.

[0068] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A building change detection method based on orthophoto and oblique photography data, characterized in that: The method comprises the following steps: Step S1: For the same area, two phases of orthophotos are obtained, and raster patches of building outline changes are obtained through building extraction and building outline change detection, which are further converted into vector patches; Step S2: Obtain oblique photography data corresponding to the two periods of orthophotos for the area, and obtain building height change vector maps by detecting building height changes; Step S3: Fusing the building outline change vector image patch and the height change vector image patch to obtain the overall building change image patch; The specific process of step S1 is as follows: S11. Train a deep learning-based building recognition model by collecting data; S12, obtaining two orthophotos of the same area, and using the building recognition model to predict the building patterns in the two orthophotos to obtain building patches; S13, using Tensor Polygonization to generate a simplified building outline vector for the building patch, and obtaining a building outline change vector patch based on the changes in the building outline vectors in the two phases; In step S11, the data collection includes collecting the required high-resolution remote sensing data, recording the shooting time and height of the high-resolution remote sensing data, preprocessing the data, and resampling it to the required resolution; in step S11, the building recognition model includes an encoding network, a decoding network, and a pixel-by-pixel classifier; In step S13, the process of generating a simplified building outline vector is as follows: Generate building heat map through building recognition model and obtain building vector; According to the principle of building shape characteristics, the building vector is smoothed to eliminate the jagged edges in the generated vector; Identify building corners and mark corner locations based on the directions generated in the building recognition model; Delete the vectors between corners and optimize the position of generated building outline vectors; The specific process of step S2 is as follows: S21, obtaining oblique photography data corresponding to the two periods of orthophotos, merging the triangulated networks of the oblique photography data to form a single large-scale oblique photography data; S22, using the building spot outline data to be superimposed on the merged oblique photography data, where the oblique photography data carries high-level information of all pixels, obtaining the highest high-level information within the oblique photography data range as a high-level value, and writing the high-level value as an attribute field into the current building spot outline data; S23. After obtaining the building outline data with high-rise values in two periods, matching is performed according to the range coordinates, and then the high-rise values are compared to obtain the height change vector map of the building whose high-rise values have changed.

Images