A method and system for extracting regular outlines of buildings based on deep learning

Abstract

The present invention discloses a method and system for extracting regular outlines of buildings based on deep learning, including: according to a preset acquisition rule, obtaining a corresponding initial building outline based on a first remote sensing image, and obtaining a corresponding minimum bounding rectangle according to the initial building outline; respectively calculating a first area of the initial building outline and a second area of the minimum bounding rectangle to obtain a first ratio between the first area and the second area; when the first ratio is greater than or equal to a first preset value, determining the minimum bounding rectangle as the final extraction result; when the first ratio is less than the first preset value, dividing a plurality of regions to be measured according to the intersection points of the initial building outline and the minimum bounding rectangle, and determining whether to remove the maximum inscribed rectangle corresponding to each region to be measured on the basis of the minimum bounding rectangle. The present invention optimizes the initial building outline based on the algorithms of the minimum bounding rectangle and the maximum inscribed rectangle to improve the accuracy of regular extraction of the building outline.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and particularly to a method and system for extracting regular outlines of buildings based on deep learning. Background Art

[0002] With the rapid progress of image sensor technology, the difficulty of obtaining remote sensing images by people has gradually decreased, and the resolution of the images is continuously improving. Satellite remote sensing images can easily achieve a ground resolution of sub-meter level. In recent years, with the rapid development of China's economy, the application scope of remote sensing images has also been continuously expanded, including multiple aspects such as land change detection, urban data update, disaster prevention and emergency, urban planning, population estimation, topographic map production and update, etc. Therefore, using remote sensing images for building recognition and extraction has become a hot topic in remote sensing science work in recent years.

[0003] Traditional building extraction methods are mainly divided into two categories: one is the building extraction algorithm based on multi-spectral remote sensing images, which mainly uses the differences in spectral reflection characteristics of different ground objects to calculate different ground objects through the bands of multi-spectral remote sensing images; the other is to extract ground objects using panchromatic images or RGB images, which mainly uses the geometric features and texture headers of the images to perform image segmentation to achieve ground object extraction. However, traditional building extraction algorithms are easily affected by various factors such as seasonal weather changes, sensor quality, and building styles, resulting in insufficient extraction accuracy and being difficult to be applied to real life. Therefore, in real life, most building extraction methods still use manual annotation, which consumes a lot of manpower, financial resources, and has a high time cost and low work efficiency. Summary of the Invention

[0004] The present invention provides a method and system for extracting regular outlines of buildings based on deep learning, which reduces the time cost and optimizes the outlines of buildings corresponding to remote sensing images to improve the accuracy of regular extraction of building outlines.

[0005] To solve the above technical problems, an embodiment of the present invention provides a method for extracting regular outlines of buildings based on deep learning, including:

[0006] According to a preset acquisition rule, based on a first remote sensing image, an initial building outline is obtained, and according to the initial building outline, a minimum bounding rectangle corresponding to the initial building outline is obtained;

[0007] The first area of the initial building outline and the second area of the minimum bounding rectangle are respectively calculated to obtain a first ratio between the first area and the second area;

[0008] When the first ratio is greater than or equal to the first preset value, determine the minimum bounding rectangle as the final extraction result of the regular building contour;

[0009] When the first ratio is less than the first preset value, divide a plurality of regions to be measured according to the intersection points of the initial building contour and the minimum bounding rectangle, calculate the third area of each region to be measured, and then determine whether to remove the maximum inscribed rectangle corresponding to each region to be measured on the basis of the minimum bounding rectangle according to each third area, so as to obtain the final extraction result of the regular building contour.

[0010] By implementing the embodiments of the present application, a fuzzy initial building contour can be obtained according to the first remote sensing image, and the initial building contour can be regularized based on the optimization algorithm of the minimum bounding rectangle and the maximum inscribed rectangle, so as to obtain a more accurate regular building contour.

[0011] Further, the determining whether to remove the maximum inscribed rectangle corresponding to each region to be measured on the basis of the minimum bounding rectangle to obtain the final extraction result of the regular building contour is specifically:

[0012] Calculate the difference between the first area and the second area to obtain a second ratio between each third area and the difference, and determine the final extraction result of the regular building contour according to the magnitude relationship between each second ratio and the second preset value;

[0013] If the current second ratio is greater than or equal to the second preset value, obtain the maximum inscribed rectangle corresponding to the current region to be measured, and remove the maximum inscribed rectangle on the basis of the current minimum bounding rectangle to obtain an updated minimum bounding rectangle;

[0014] If the current second ratio is less than the second preset value, retain the current region to be measured on the basis of the current minimum bounding rectangle to obtain an updated minimum bounding rectangle;

[0015] After completing the processing of all regions to be measured and obtaining the updated minimum bounding rectangle, take the current minimum bounding rectangle as the final extraction result of the regular building contour.

[0016] By implementing the embodiments of the present application, it is possible to judge whether the area of the current region to be measured is lower than the preset acceptable range through the magnitude relationship between the second ratio and the second preset value, that is, whether the current region to be measured can be ignored within a certain range and does not affect the final extraction result of the regular building contour, thereby improving the accuracy of the regular extraction of the building contour.

[0017] Further, according to the preset acquisition rule, based on the first remote sensing image, the corresponding initial building contour is acquired, and according to the initial building contour, the minimum bounding rectangle corresponding to the initial building contour is obtained, specifically as follows:

[0018] The first remote sensing image is input into the segmentation model to extract the features of the first remote sensing image, and the first feature map corresponding to the first remote sensing image is obtained;

[0019] Perform multi-semantic multi-scale fusion on the first feature map to obtain the second feature map corresponding to the first feature map;

[0020] According to the preset rule, perform prediction on the second feature map to obtain the corresponding first binary image;

[0021] Perform filtering preprocessing of erosion and dilation on the first binary image in sequence to obtain the corresponding second binary image; wherein, the second binary image contains the initial building contour;

[0022] According to the initial building contour, obtain the minimum bounding rectangle corresponding to the initial building contour.

[0023] Implementing the embodiments of the present application, the features of the first remote sensing image can be extracted through the segmentation model, and the extracted features can be predicted to implement the segmentation processing of the first remote sensing image and obtain the corresponding first binary image. After obtaining the first binary image, filtering preprocessing of erosion and dilation can also be performed on the first binary image in sequence to eliminate the interference noise in the first binary image and fill its hole points, and finally obtain the corresponding second binary image.

[0024] Further, before the first remote sensing image is input into the segmentation model to extract the features of the first remote sensing image and obtain the first feature map corresponding to the first remote sensing image, it further includes:

[0025] Obtain a plurality of second remote sensing images, and respectively obtain the binary images corresponding to the second remote sensing images according to the second remote sensing images; wherein, one second remote sensing image corresponds to one binary image;

[0026] Randomly crop each of the second remote sensing images and each of the binary images according to the preset sample image parameters to obtain a plurality of sample images corresponding to the second remote sensing images, and a plurality of image labels corresponding to the binary images, and perform data augmentation processing on all the sample images and all the image labels to obtain a training dataset; wherein, one sample image corresponds to one image label;

[0027] Build a deep neural network model, and use the training dataset to train the deep neural network model to obtain a segmentation model.

[0028] By implementing the embodiments of the present application, each of the second remote sensing images and each of the binary images can be randomly cropped to obtain a number of sample images and corresponding image labels, and data augmentation processing can be performed on the number of sample images and corresponding image labels that make up the training dataset to enrich the data quantity and data type of the training dataset, so that the trained segmentation model can adapt to various types of remote sensing images and enhance the generalization ability of the model.

[0029] To solve the same technical problem, the present invention also provides an extraction system for regular building outlines based on deep learning, including:

[0030] A preprocessing module for obtaining a corresponding initial building outline according to the first remote sensing image according to a preset acquisition rule, and obtaining a minimum bounding rectangle corresponding to the initial building outline according to the initial building outline;

[0031] A calculation module for respectively calculating a first area of the initial building outline and a second area of the minimum bounding rectangle to obtain a first ratio between the first area and the second area;

[0032] A first contour extraction module for determining the minimum bounding rectangle as the final extraction result of the regular building outline when the first ratio is greater than or equal to a first preset value;

[0033] A second contour extraction module for, when the first ratio is less than the first preset value, dividing a plurality of regions to be measured according to the intersection points of the initial building outline and the minimum bounding rectangle, calculating a third area of each of the regions to be measured, and then determining whether to remove the maximum inscribed rectangle corresponding to each of the regions to be measured on the basis of the minimum bounding rectangle to obtain the final extraction result of the regular building outline.

[0034] Further, the second contour extraction module further includes:

[0035] A data processing unit for calculating the difference between the first area and the second area to obtain a second ratio between each of the third areas and the difference;

[0036] A first contour extraction unit for, if the current second ratio is greater than or equal to a second preset value, obtaining the maximum inscribed rectangle corresponding to the current region to be measured and removing the maximum inscribed rectangle on the basis of the current minimum bounding rectangle to obtain an updated minimum bounding rectangle;

[0037] A second contour extraction unit, configured to, if the current second ratio is less than a second preset value, retain the current area to be measured on the basis of the current minimum bounding rectangle, and obtain an updated minimum bounding rectangle;

[0038] A result acquisition unit, configured to, after completing the processing of all the areas to be measured and obtaining the updated minimum bounding rectangle, use the current minimum bounding rectangle as the final extraction result of the regular building contour.

[0039] Further, the preprocessing module further includes:

[0040] A feature extraction unit, configured to input the first remote sensing image into a segmentation model, extract the features of the first remote sensing image, and obtain a first feature map corresponding to the first remote sensing image;

[0041] A feature fusion unit, configured to perform multi-semantic multi-scale fusion on the first feature map to obtain a second feature map corresponding to the first feature map;

[0042] A prediction unit, configured to predict the second feature map according to a preset rule to obtain a corresponding first binary image;

[0043] A preprocessing unit, configured to perform filtering preprocessing of erosion and dilation on the first binary image in sequence to obtain a corresponding second binary image; wherein, the second binary image contains the initial building contour;

[0044] A third contour extraction unit, configured to obtain a minimum bounding rectangle corresponding to the initial building contour according to the initial building contour.

[0045] Further, the extraction system of the regular building contour based on deep learning further includes:

[0046] A model training module, configured to obtain a plurality of second remote sensing images, and respectively obtain binary images corresponding to the second remote sensing images according to the second remote sensing images; wherein, one second remote sensing image corresponds to one binary image; randomly crop the second remote sensing images and the binary images respectively according to preset sample image parameters to obtain a plurality of sample images corresponding to the second remote sensing images, and a plurality of image labels corresponding to the binary images, and perform data augmentation processing on all the sample images and all the image labels to obtain a training data set; wherein, one sample image corresponds to one image label; construct a deep neural network model, and use the training data set to train the deep neural network model to obtain a segmentation model.

[0047] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0048] The present invention provides a method and system for extracting regular outlines of buildings based on deep learning. By obtaining the minimum bounding rectangle corresponding to the initial building outline, the rough initial building outline in the first remote sensing image is preliminarily regularized. Moreover, when the area difference between the minimum bounding rectangle and the initial building outline is large, multiple regions to be measured are divided according to the intersection points of the minimum bounding rectangle and the initial building outline, and it is determined whether to remove the maximum inscribed rectangle corresponding to the region to be measured based on the minimum bounding rectangle, further optimizing the outline of the building in the first remote sensing image and improving the accuracy of regularized extraction of the building outline.

[0049] Furthermore, the present invention also performs segmentation processing on the first remote sensing image using a segmentation model, and performs filtering preprocessing of erosion and dilation on the segmentation result to eliminate interference noise, fill in hole points, optimize the initial data, and further improve the accuracy of extracting the regular outline of the building; among them, the segmentation model is obtained by training a deep neural network model using a training data set, enabling the segmentation model to adapt to various types of remote sensing images and enhancing the generalization ability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 : is a schematic flow chart of a method for extracting regular outlines of buildings based on deep learning provided by an embodiment of the present invention;

[0051] Figure 2 : is a schematic flow chart of a method for obtaining a first binary image provided by an embodiment of the present invention;

[0052] Figure 3 : is a training sample obtained by splicing provided by an embodiment of the present invention;

[0053] Figure 4 : is a schematic flow chart of regularizing the building outline based on the minimum bounding rectangle and the maximum inscribed rectangle provided by an embodiment of the present invention;

[0054] Figure 5 : is a schematic structural diagram of a system for extracting regular outlines of buildings based on deep learning provided by an embodiment of the present invention;

[0055] Figure 6 : is a schematic structural diagram of a preprocessing module of a system for extracting regular outlines of buildings based on deep learning provided by an embodiment of the present invention;

[0056] Figure 7 : is a schematic structural diagram of a second contour extraction module of a system for extracting regular outlines of buildings based on deep learning provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0057] Next, in combination with the accompanying drawings in the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without making creative efforts belong to the scope of protection of the present invention.

[0058] Embodiment 1:

[0059] Please refer to Figure 1 , a method for extracting the regular contour of a building based on deep learning provided by an embodiment of the present invention. This method includes steps S1 to S4, and the specific steps are as follows:

[0060] Step S1: According to a preset acquisition rule, based on the first remote sensing image, obtain the corresponding initial building contour, and based on the initial building contour, obtain the minimum bounding rectangle corresponding to the initial building contour.

[0061] Further, step S1 specifically includes steps S11 to S15, and the specific steps are as follows:

[0062] Step S11: Input the first remote sensing image into the segmentation model, extract the features of the first remote sensing image, and obtain the first feature map corresponding to the first remote sensing image.

[0063] In this embodiment, please refer to Figure 2 , the segmentation model selects ResNet50 to extract the features of the input first remote sensing image, and obtain the corresponding first feature map (i.e., FeatureMap). Among them, the size of the first feature map is 1 / 8 of the size of the corresponding first remote sensing image.

[0064] Step S12: Perform multi-semantic multi-scale fusion on the first feature map to obtain the second feature map corresponding to the first feature map.

[0065] In this embodiment, the process of multi-semantic multi-scale fusion in the segmentation model is specifically as follows:

[0066] First, please refer to Figure 2 , and extract different features of the first feature map through the following four branches respectively.

[0067] The first branch: Perform global average pooling operation on the first feature map to obtain bin_1 with a size of 1×1;

[0068] The second branch: Divide the first feature map into 2×2 sub-regions, and then perform global average pooling operation on each sub-region respectively to obtain bin_2 with a size of 2×2;

[0069] Third branch: Divide the first feature map into 3×3 sub-regions, and then perform average pooling operations on each sub-region respectively to obtain bin_3 with a size of 3×3;

[0070] Fourth branch: Divide the feature image into 6×6 sub-regions, and then perform average pooling operations on each sub-region respectively to obtain bin_4 with a size of 6×6;

[0071] Secondly, perform 1×1 convolution operations on bin_1, bin_2, bin_3, and bin_4 respectively to reduce the output dimension to 1 / 4 of the original input dimension, and perform bilinear interpolation upsampling operations on bin_1, bin_2, bin_3, and bin_4 respectively to obtain the corresponding first branch image, second branch image, third branch image, and fourth branch image. Among them, the resolution sizes of the first branch image, second branch image, third branch image, and fourth branch image are the same as the resolution size of the first feature map.

[0072] Finally, adopt the channel aggregation method. On the basis of the first feature map, stack the first branch image, second branch image, third branch image, and fourth branch image in turn along the channel axis to finally form the second feature map corresponding to the first feature map.

[0073] Step S13: Predict the second feature map according to a preset rule to obtain the corresponding first binary image.

[0074] In this embodiment, please refer to Figure 2 , and through the segmentation model, predict (Final Prediction) the second feature map obtained by completing multi-semantic multi-scale fusion to obtain the corresponding first binary image.

[0075] Step S14: Perform filtering preprocessing of erosion and dilation on the first binary image in turn to obtain the corresponding second binary image; among them, the second binary image contains the initial building contour.

[0076] In this embodiment, the erosion and dilation operations belong to morphological filtering processing, which is used to remove interference noise. Specifically: erode the first binary image to eliminate the isolated point noise in the first binary image; and perform dilation operations on the first binary image that has completed the erosion operation to fill small holes.

[0077] Among them, the erosion and dilation operations use a rectangular structural element with a size of 3×3 to construct the operation convolution kernel.

[0078] Step S15: Obtain the minimum bounding rectangle corresponding to the initial building contour according to the initial building contour.

[0079] Further, before step S11, steps S01 to S03 are further included, and the specific steps are as follows:

[0080] Step S01: Obtain multiple second remote sensing images, and respectively obtain binary images corresponding to the second remote sensing images according to the second remote sensing images; where one second remote sensing image corresponds to one binary image.

[0081] In this embodiment, high-definition second remote sensing images in tif format are obtained from BigeMap, and the high-definition second remote sensing images are imported into the ArcGis software to produce a vector file in shp format of building outlines. Then, the mask extraction tool in the ArcGis software is used to extract and analyze the vector file in shp format of building outlines, and then the corresponding building mask image is obtained. Finally, through image processing, the building mask image is converted into a corresponding binary image. Among them, for the binary image, the value of 0 represents the background, and the value of 255 represents the building.

[0082] Step S02: Randomly crop each second remote sensing image and each binary image according to preset sample image parameters to obtain sample images corresponding to a number of second remote sensing images and image labels corresponding to a number of binary images, and perform data augmentation processing on all sample images and all image labels to obtain a training dataset; where one sample image corresponds to one image label.

[0083] In this embodiment, since the process of data acquisition is random, the sizes of all the second remote sensing images and the corresponding binary images obtained are not uniform, which will affect the subsequent training effect. Therefore, before training the initial model, parameters such as the height and width of the sample images are preset according to the processing ability of the actual graphics card, and according to the preset parameters, random cropping is performed on each second remote sensing image to obtain sample images corresponding to a number of second remote sensing images. Similarly, random cropping is performed on each binary image to obtain image labels corresponding to a number of binary images.

[0084] As an example, set the height of the sample image Image_height = 512, the width of the sample image Image_width = 512, and then randomly crop sample images with a size of 512×512 on each second remote sensing image, and randomly crop image labels with a size of 512×512 on each binary image.

[0085] As can be seen from the above, the sample images and image labels obtained by each cropping correspond to different regions of the second remote sensing images and the binary images, that is, it is ensured that the sample images and image labels obtained by each cropping have spatial randomness. Among them, the sizes of all sample images and all image labels are consistent, and the image label is a representation of the location of the building in the corresponding sample image.

[0086] To further enrich the training dataset and expand the coverage of the training dataset, data augmentation is performed on all sample images and all image labels to obtain the training dataset. Specifically, data augmentation such as left - right flipping, saturation and chroma stretching, noise addition, blurring, and 45 - degree clockwise and counter - clockwise tilting can be performed on the sample images. Similarly, the above - mentioned data augmentation is performed on the image labels.

[0087] In addition, in addition to the above - mentioned conventional data augmentation, the mix - moasic data augmentation method can also be used to perform data augmentation on all sample images and all image labels, so as to significantly improve the building semantic segmentation ability of the deep neural network model for ultra - large high - definition remote sensing images and significantly improve the segmentation accuracy. Specifically, the mix - moasic data augmentation method is to randomly select four training samples from all the training datasets, randomly scale each training sample by a certain ratio, and finally splice the four scaled training samples into one training sample. Among them, the scaling ratio of each training sample is random, that is, the sizes of the four scaled training samples are not necessarily the same, but the training sample obtained by splicing the four scaled training samples has the same size as the original training sample, that is, the same as the preset sample image parameter size. At the same time, referring to Figure 3 , the training sample obtained by splicing has obvious boundary lines. Therefore, the mix - moasic data augmentation method not only increases the number of training samples, but also can increase the styles of image splicing, enabling the trained segmentation model to adapt to various obvious splicing traces in high - definition remote sensing images and greatly enhancing the generalization ability of the model.

[0088] Step S03: Construct a deep neural network model and use the training dataset to train the deep neural network model to obtain a segmentation model.

[0089] In this embodiment, based on the constructed deep neural network model, the training dataset obtained after data augmentation is input into the deep neural network model for model training to update the network weights of the deep neural network model, further enhancing the generalization ability of the deep neural network model and improving the accuracy of the model. Among them, when constructing the deep neural network model, the initial settings are as follows: the initial weight is the imagenet network weight, the initial learning rate is lr, the method for updating the learning rate is the exponential decay method, the method for the network to update weights is the adam method, and the method for calculating the total loss value is the binary cross - entropy loss function.

[0090] Specifically, after the data in the training data set is batch-inputted into the deep neural network model, the deep neural network model extracts features from the data and obtains the corresponding feature map, and then updates the feature map containing multi-scale multi-semantic fusion through multi-semantic multi-scale fusion operations, and extracts the regions of interest and non-regions of interest in the updated feature map accordingly. Then, based on the extracted regions of interest and non-regions of interest, the total loss value output by the deep neural network model is calculated, and the deep neural network weights are back-propagated and updated with the total loss value to obtain the segmentation model after training.

[0091] Step S2: Calculate a first area area_a of the initial building outline and a second area area_b of the minimum circumscribed rectangle respectively, divide area_a by area_b and multiply by 100% to obtain a first ratio ratio_1 between the first area area_a and the second area area_b.

[0092] It should be noted that when the first ratio ratio_1 is greater than or equal to the first preset value, step S3 is executed; when the first ratio ratio_1 is less than the first preset value, step S4 is executed. As an example, ratio_1=80%, and the value of ratio_1 is obtained by multiple experiments, which can be adjusted according to the actual situation, so that the final building regular contour extraction result is less different from the first area area_a of the initial building contour, so as to ensure the accuracy of the building contour regularization extraction.

[0093] Step S3: Determine the minimum bounding rectangle as the final building regular outline extraction result.

[0094] Step S4: Divide a plurality of test areas according to the intersection of the initial building outline and the minimum circumscribed rectangle, and calculate the third area of ​​each test area. Then, based on each third area, determine whether to eliminate the maximum inscribed rectangle corresponding to each test area on the basis of the minimum circumscribed rectangle to obtain the final building regular outline extraction result.

[0095] Further, step S4 specifically includes step S41 to step S45, and each step is specifically as follows:

[0096] Step S41: Divide a plurality of areas to be measured according to the intersection of the initial building outline and the minimum circumscribed rectangle, and calculate the third area area_c of each area to be measured.

[0097] As an example, see Figure 4, take the upper left vertex o of the minimum circumscribed rectangle as the coordinate origin, and the two adjacent vertical sides of the upper left vertex o of the minimum circumscribed rectangle as the coordinate axes, establish a new coordinate system, and assume that the four intersection points of the initial building outline and the minimum circumscribed rectangle are a, b, c, and d. At this time, the edge of the initial building outline, the minimum circumscribed rectangle and the intersection points a, b, c, and d respectively form four areas to be measured 1, 2, 3, and 4, and calculate the third area area_c of each area to be measured.

[0098] Among them, the initial building outline obtained is different according to different remote sensing images, and the number of intersections between the initial building outline and the minimum circumscribed rectangle will also change accordingly. Therefore, the assumptions about the intersections and the division of the area to be measured will also be adaptively adjusted.

[0099] Step S42: Calculate the difference area_delta between the first area area_a and the second area area_b, divide area_c by area_delta and multiply by 100% to obtain the second ratio ratio_2 between each third area area_c and the difference area_delta, and determine the final building regular outline extraction result according to the size relationship between each second ratio ratio_2 and the second preset value.

[0100] It should be noted that if the second ratio ratio_2 is greater than or equal to the second preset value, step S43 is executed; if the second ratio ratio_2 is less than the second preset value, step S44 is executed. As an example, ratio_2=25%, and the value of ratio_2 is obtained by multiple experiments, which can be adjusted according to the actual situation, so that the final building regular contour extraction result is less different from the first area area_a of the initial building contour, further ensuring the accuracy of the building contour regular extraction.

[0101] Step S43: obtaining the maximum inscribed rectangle corresponding to the current area to be measured, and eliminating the maximum inscribed rectangle based on the current minimum circumscribed rectangle to obtain an updated minimum circumscribed rectangle.

[0102] As an example, when the second ratio ratio_2 is greater than or equal to the second preset value, the maximum inscribed rectangle of the test area 1 is obtained by using the dynamic programming method. Figure 4, one vertex o of the largest inscribed rectangle is the upper left vertex o of the smallest circumscribed rectangle. The two adjacent vertical sides of the vertex o of the largest inscribed rectangle are the two vertical sides where the intersection points a and b of the smallest circumscribed rectangle are located. The intersection point of the largest inscribed rectangle and the ab segment of the initial building outline is another vertex e that is not adjacent to the vertex o. It is assumed that the other two vertices of the largest inscribed rectangle are f and g respectively. Finally, the aeb curve in the initial building outline is replaced by the line segment afegb (that is, the largest inscribed rectangle ofeg is removed on the basis of the current smallest circumscribed rectangle), and the updated smallest circumscribed rectangle is obtained.

[0103] Among them, the dynamic programming method is specifically as follows: taking o as the vertex, the side length in the reverse direction of oa is set to 1 respectively, and the side length in the ob direction is set to 1. Then, the side lengths in the oa direction and the ob direction are continuously increased to calculate the area of the rectangle until the rectangle with the largest area is found, which is the largest inscribed rectangle.

[0104] Step S44: On the basis of the current smallest circumscribed rectangle, retain the current area to be measured, and obtain the updated smallest circumscribed rectangle.

[0105] As an example, when the second ratio ratio_2 is less than the second preset value, retain the current area to be measured on the basis of the current smallest circumscribed rectangle. Please refer to Figure 4 , for example, for the area to be measured 3, directly use the line segment edge of the area to be measured 3 to replace the corresponding curve edge in the initial building outline, and obtain the updated smallest circumscribed rectangle.

[0106] Step S45: After completing the processing of all areas to be measured and obtaining the updated smallest circumscribed rectangle, use the current smallest circumscribed rectangle as the final result of the regular building outline extraction.

[0107] As an example, after completing the processing of all areas to be measured and obtaining the updated smallest circumscribed rectangle, the obtained result is shown in Figure 4 .

[0108] To solve the same technical problem, please refer to Figure 5 , the present invention also provides an extraction system for the regular building outline based on deep learning, including:

[0109] A preprocessing module 1, configured to obtain the corresponding initial building outline according to the first remote sensing image according to a preset acquisition rule, and obtain the smallest circumscribed rectangle corresponding to the initial building outline according to the initial building outline;

[0110] A calculation module 2, configured to calculate the first area of the initial building outline and the second area of the smallest circumscribed rectangle respectively to obtain the first ratio between the first area and the second area;

[0111] The first contour extraction module 3 is used to determine the minimum bounding rectangle as the final extraction result of the regular building contour when the first ratio is greater than or equal to the first preset value;

[0112] The second contour extraction module 4 is used to, when the first ratio is less than the first preset value, divide a plurality of regions to be measured according to the intersection points of the initial building contour and the minimum bounding rectangle, calculate the third area of each region to be measured, and then determine whether to remove the maximum inscribed rectangle corresponding to each region to be measured on the basis of the minimum bounding rectangle according to each third area, so as to obtain the final extraction result of the regular building contour.

[0113] Furthermore, please refer to Figure 5 , the extraction system of the regular building contour based on deep learning further includes:

[0114] The model training module 5 is used to obtain a plurality of second remote sensing images, and respectively obtain the binary images corresponding to the second remote sensing images according to the second remote sensing images; wherein, one second remote sensing image corresponds to one binary image; randomly crop each second remote sensing image and each binary image according to the preset sample image parameters to obtain a plurality of sample images corresponding to the second remote sensing images and a plurality of image labels corresponding to the binary images, and perform data augmentation processing on all the sample images and all the image labels to obtain a training data set; wherein, one sample image corresponds to one image label; construct a deep neural network model, and use the training data set to train the deep neural network model to obtain a segmentation model.

[0115] Furthermore, please refer to Figure 6 , the preprocessing module further includes:

[0116] The feature extraction unit is used to input the first remote sensing image into the segmentation model, extract the features of the first remote sensing image, and obtain the first feature map corresponding to the first remote sensing image;

[0117] The feature fusion unit is used to perform multi-semantic multi-scale fusion on the first feature map to obtain the second feature map corresponding to the first feature map;

[0118] The prediction unit is used to predict the second feature map according to the preset rules to obtain the corresponding first binary image;

[0119] The preprocessing unit is used to perform filtering preprocessing of erosion and dilation on the first binary image in sequence to obtain the corresponding second binary image; wherein, the second binary image contains the initial building contour;

[0120] The third contour extraction unit is used to obtain the minimum bounding rectangle corresponding to the initial building contour according to the initial building contour.

[0121] Further, please refer to Figure 7 , the second contour extraction module further includes:

[0122] A data processing unit, configured to calculate the difference between the first area and the second area, and calculate the second ratio between each third area and the difference;

[0123] A first contour extraction unit, configured to, if the current second ratio is greater than or equal to a second preset value, obtain the maximum inscribed rectangle corresponding to the current area to be measured, and remove the maximum inscribed rectangle on the basis of the current minimum circumscribed rectangle to obtain an updated minimum circumscribed rectangle;

[0124] A second contour extraction unit, configured to, if the current second ratio is less than the second preset value, retain the current area to be measured on the basis of the current minimum circumscribed rectangle to obtain an updated minimum circumscribed rectangle;

[0125] A result acquisition unit, configured to, after completing the processing of all areas to be measured and obtaining the updated minimum circumscribed rectangle, use the current minimum circumscribed rectangle as the final result of the regular contour extraction of the building.

[0126] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working process of the above-described system can refer to the corresponding process in the foregoing method embodiment, and will not be described herein again.

[0127] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0128] The present invention provides a method and system for extracting a regular contour of a building based on deep learning. By obtaining the minimum circumscribed rectangle corresponding to the initial building contour, the rough initial building contour in the first remote sensing image is preliminarily regularized. Moreover, when the area difference between the minimum circumscribed rectangle and the initial building contour is large, multiple areas to be measured are divided according to the intersection points of the minimum circumscribed rectangle and the initial building contour, and it is determined whether to remove the maximum inscribed rectangle corresponding to the area to be measured on the basis of the minimum circumscribed rectangle, further optimizing the contour of the building in the first remote sensing image and improving the accuracy of the regular contour extraction of the building.

[0129] Further, the present invention also uses a segmentation model to perform segmentation processing on the first remote sensing image, and performs corrosion and dilation filtering preprocessing on the segmentation result to eliminate interference noise, fill hole points, optimize the initial data, and further improve the accuracy of the regular contour extraction of the building; wherein, the segmentation model is obtained by training a deep neural network model using a training data set, so that the segmentation model can adapt to various types of remote sensing images and enhance the generalization ability of the model.

[0130] The specific embodiments described above further elaborate on the purpose, technical solutions, and beneficial effects of the present invention. It should be understood that the above are only specific embodiments of the present invention and are not used to limit the protection scope of the present invention. In particular, for those skilled in the art, any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for extracting regular outlines of buildings based on deep learning, characterized in that, Including: According to a preset acquisition rule, based on the first remote sensing image, obtain the corresponding initial building contour, and based on the initial building contour, obtain the minimum bounding rectangle corresponding to the initial building contour; Calculate the first area of the initial building contour and the second area of the minimum bounding rectangle respectively to obtain the first ratio between the first area and the second area; When the first ratio is greater than or equal to a first preset value, determine the minimum bounding rectangle as the final result of the regular building contour extraction; When the first ratio is less than the first preset value, divide multiple regions to be measured according to the intersection points of the initial building contour and the minimum bounding rectangle, calculate the third area of each region to be measured, and then determine whether to remove the maximum inscribed rectangle corresponding to each region to be measured based on the minimum bounding rectangle according to each third area, so as to obtain the final result of the regular building contour extraction; The step of determining whether to remove the maximum inscribed rectangle corresponding to each region to be measured based on the minimum bounding rectangle to obtain the final result of the regular building contour extraction is specifically: Calculate the difference between the first area and the second area to obtain the second ratio between each third area and the difference, and determine the final result of the regular building contour extraction according to the magnitude relationship between each second ratio and a second preset value; If the current second ratio is greater than or equal to the second preset value, obtain the maximum inscribed rectangle corresponding to the current region to be measured, and remove the maximum inscribed rectangle based on the current minimum bounding rectangle to obtain an updated minimum bounding rectangle; If the current second ratio is less than the second preset value, retain the current region to be measured based on the current minimum bounding rectangle to obtain an updated minimum bounding rectangle; After completing the processing of all regions to be measured and obtaining the updated minimum bounding rectangle, use the current minimum bounding rectangle as the final result of the regular building contour extraction.

2. The method for extracting regular outlines of buildings based on deep learning according to claim 1, characterized in that, The step of according to a preset acquisition rule, based on the first remote sensing image, obtaining the corresponding initial building contour, and based on the initial building contour, obtaining the minimum bounding rectangle corresponding to the initial building contour is specifically: Input the first remote sensing image into a segmentation model, extract the features of the first remote sensing image, and obtain the first feature map corresponding to the first remote sensing image; Perform multi-semantic multi-scale fusion on the first feature map to obtain the second feature map corresponding to the first feature map; Predict the second feature map according to a preset rule to obtain the corresponding first binary image; Perform filtering preprocessing of erosion and dilation on the first binary image in sequence to obtain the corresponding second binary image; wherein, the second binary image contains the initial building contour; Based on the initial building contour, obtain the minimum bounding rectangle corresponding to the initial building contour.

3. The method for extracting regular outlines of buildings based on deep learning according to claim 2, characterized in that, Before the step of inputting the first remote sensing image into a segmentation model, extracting the features of the first remote sensing image, and obtaining the first feature map corresponding to the first remote sensing image, it further includes: Obtain multiple second remote sensing images, and respectively obtain binary images corresponding to each of the second remote sensing images according to each of the second remote sensing images; wherein, one second remote sensing image corresponds to one binary image. Randomly crop each of the second remote sensing images and each of the binary images according to preset sample image parameters to obtain a number of sample images corresponding to the second remote sensing images and a number of image labels corresponding to the binary images, and perform data augmentation processing on all the sample images and all the image labels to obtain a training data set; wherein, one sample image corresponds to one image label. Construct a deep neural network model, and use the training data set to train the deep neural network model to obtain a segmentation model.

4. A system for extracting regular outlines of buildings based on deep learning, characterized in that, Comprising: A preprocessing module, configured to obtain a corresponding initial building contour according to a first remote sensing image according to a preset acquisition rule, and obtain a minimum bounding rectangle corresponding to the initial building contour according to the initial building contour. A calculation module, configured to calculate a first area of the initial building contour and a second area of the minimum bounding rectangle respectively to obtain a first ratio between the first area and the second area. A first contour extraction module, configured to determine the minimum bounding rectangle as the final building regular contour extraction result when the first ratio is greater than or equal to a first preset value. A second contour extraction module, configured to, when the first ratio is less than the first preset value, divide a plurality of regions to be measured according to the intersection points of the initial building contour and the minimum bounding rectangle, calculate a third area of each of the regions to be measured, and then determine whether to remove the maximum inscribed rectangle corresponding to each of the regions to be measured on the basis of the minimum bounding rectangle according to the third areas of each of the regions to be measured to obtain the final building regular contour extraction result. The second contour extraction module further includes: A data processing unit, configured to calculate a difference between the first area and the second area to obtain a second ratio between each of the third areas and the difference. A first contour extraction unit, configured to, if the current second ratio is greater than or equal to a second preset value, obtain the maximum inscribed rectangle corresponding to the current region to be measured, and remove the maximum inscribed rectangle on the basis of the current minimum bounding rectangle to obtain an updated minimum bounding rectangle. A second contour extraction unit, configured to, if the current second ratio is less than the second preset value, retain the current region to be measured on the basis of the current minimum bounding rectangle to obtain an updated minimum bounding rectangle. A result acquisition unit, configured to, after completing the processing of all the regions to be measured and obtaining the updated minimum bounding rectangle, use the current minimum bounding rectangle as the final building regular contour extraction result.

5. The system for extracting regular outlines of buildings based on deep learning according to claim 4, characterized in that, The preprocessing module further includes: A feature extraction unit, configured to input the first remote sensing image into the segmentation model, extract features of the first remote sensing image, and obtain a first feature map corresponding to the first remote sensing image. A feature fusion unit for performing multi-semantic and multi-scale fusion on the first feature map to obtain a second feature map corresponding to the first feature map; A prediction unit for predicting the second feature map according to a preset rule to obtain a corresponding first binary image; A preprocessing unit for sequentially performing erosion and dilation filtering preprocessing on the first binary image to obtain a corresponding second binary image; wherein, the second binary image contains the initial building contour; A third contour extraction unit for obtaining a minimum bounding rectangle corresponding to the initial building contour according to the initial building contour.

6. The extraction system of the regular outline of a building based on deep learning according to claim 4, characterized in that, Further comprising: A model training module for obtaining a plurality of second remote sensing images, and respectively obtaining binary images corresponding to the second remote sensing images according to the second remote sensing images; wherein, one second remote sensing image corresponds to one binary image; randomly cropping each of the second remote sensing images and each of the binary images according to preset sample image parameters to obtain a plurality of sample images corresponding to the second remote sensing images, and a plurality of image labels corresponding to the binary images, and performing data augmentation processing on all the sample images and all the image labels to obtain a training data set; wherein, one sample image corresponds to one image label; constructing a deep neural network model, and training the deep neural network model by using the training data set to obtain a segmentation model.

Images