A method for automatically extracting building boundaries based on high-resolution remote sensing images

Abstract

The application discloses a kind of building boundary automatic extraction method based on high-resolution remote sensing image, comprising the following steps: S1, for relevant building data set is filtered;S2, using dynamic scale training high-resolution network extracts the building boundary in optical remote sensing image;S3, using relevant evaluation parameters is quantified to the extraction accuracy;S4, by evaluation parameter and data set difference, the performance of method is compared and evaluated.The application, in model training stage, has applied dynamic scale training DST, to alleviate the negative influence of scale difference on performance;A scale-aware backbone, i.e. high-resolution network, is used to enhance feature representation;Finally, high-resolution supervision target is used to further improve performance.

Description

Technical Field

[0001] This invention relates to the field of optical remote sensing technology, specifically a method for automatically extracting building boundaries based on high-resolution remote sensing images. Background Technology

[0002] High-resolution building maps are fundamental data for urban applications and natural disaster management, and are of great significance to these fields. Aerial imagery is widely used for building outline extraction due to its large coverage area, high accessibility, and high level of detail. However, occlusion by trees and taller buildings, scale variations, or intra-class variations inhibit its practical application in extracting building outlines from aerial imagery. With the development of remote sensing and deep learning technologies, automatically extracting building outlines from high-resolution remote sensing imagery has gradually become feasible and has attracted widespread attention in recent years. To create these maps, building boundary delineation (or building polygon prediction) is used to obtain regular and accurate building boundaries.

[0003] In recent years, deep convolutional neural networks (DCNNs) have dominated computer vision tasks and have been successfully applied in the field of remote sensing. Instance segmentation and semantic segmentation methods based on DCNNs have been widely used to extract building outlines from aerial images. Furthermore, with the emergence of a large number of aerial images with building labels, major problems such as scale differences and intra-class differences can gradually be solved. Therefore, these methods have achieved high accuracy, but the occlusion problem remains insufficiently addressed. Additionally, blurred boundaries still exist in the extracted results.

[0004] Building contour extraction based on end-to-end DCNNs has recently attracted widespread attention. These methods were first introduced to directly generate vector building maps from remotely sensed images without any post-processing. Recent research has employed advanced techniques such as Convolutional Gate Recurrent Units (convGRU), Frame Field Learning (FFL), and Graph Neural Networks (GNNs), and novel objectives such as vertices, frame fields, attraction field maps, and permutation matrices trained through supervised modeling. Furthermore, by directly outputting vector results with building corners, the occlusion and boundary blurring problems have been significantly alleviated. Nevertheless, the scale discrepancy problem remains. However, the scale discrepancy problem limits the overall performance of the recognition. Summary of the Invention

[0005] The purpose of this invention is to provide an automatic building boundary extraction method based on high-resolution remote sensing imagery to solve the problems in the prior art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: an automatic building boundary extraction method based on high-resolution remote sensing imagery, comprising the following steps:

[0007] S1. Filter the relevant building dataset;

[0008] S2. Use a high-resolution network trained with dynamic scaling to extract building boundaries from optical remote sensing images;

[0009] S3. Use relevant evaluation parameters to quantitatively define the extraction accuracy;

[0010] S4. Compare and evaluate the method's performance by evaluating the differences between the evaluation parameters and the dataset.

[0011] Preferably, step S1 specifically includes the following steps:

[0012] S11. Select four public building datasets to broadly evaluate the performance of the inventive network and test its robustness; the datasets should meet the requirement of being widely used for building outline extraction.

[0013] S12. The filtered datasets include the AICrowd architecture dataset, the Inria architecture dataset, the WHU architecture dataset, and the Waterloo architecture dataset.

[0014] S13. The imagery in the dataset includes three bands: red, green, and blue, with different spatial resolutions of 0.3m, 0.075m, and 0.12m, respectively, and the geographical coverage areas vary.

[0015] Preferably, step S2 specifically includes the following steps:

[0016] S21. A novel HiSup-based method was developed to mitigate the scale difference problem: Mask-and-Vertices Attraction used in HiSup was applied; predicted vertices and masks were used as input to initialize polygons; local nonmaximum suppression was applied to sparse vertices; with the help of predicted offset vectors, refined vertices were used to simplify the initialized polygons, removing redundant and low-confidence vertices from the initialized polygons; if adjacent edges in each polygon are nearly parallel, they are further merged.

[0017] S22. A Dynamic Scale Training (DST) strategy was added to the model optimization. DST overcomes scaling differences by applying tiling images and supervised objectives. The supervised objectives are guided by dynamic feedback. The feedback is the proportion of the loss of small objects to all objects.

[0018] S23. The original high-resolution network (HRNet)v2 was replaced with a scale-aware high-resolution network. HRNet showed excellent performance in feature extraction and representation using multi-level features, with repeated information exchange at each stage. However, in the final stage of HRNet, the highest resolution of the features was 1 / 4 of the input value. Information loss and scale difference suppressed the performance of HRNet. HigherHRNet was proposed by adding a scale-aware module to HRNet. The scale-aware module mainly consists of a deconvolution module and 4 residual blocks. Two features with different spatial resolutions are downsampled and concatenated into 128×128.

[0019] Preferably, step S3 specifically includes the following steps:

[0020] S31. Input step S1 to filter high-resolution remote sensing images of the building dataset;

[0021] S32. Use the scale-aware high-resolution network from step S2 to train the image dataset and extract the building outlines from the images.

[0022] S33. Use rank-level evaluation metrics for instance segmentation and object detection in computer vision and remote sensing applications, including mean precision (AP), mean recall (AR), AP50, and AP75.

[0023] S34. In addition to conventional evaluation metrics, AP-Small (APs), AP-Medium (APm), AP-Large (APL), AR-Small (ARs), AR-Medium (ARm), and AR-Large (ARL) were also used as evaluation metrics to assess the extraction results and algorithm performance.

[0024] S35. A constraint metric, APboundary, has been added as an evaluation metric during the internal evaluation process. APboundary is the average accuracy calculated based on the boundary intersection unit (IoU). The specific steps for solving the boundary IoU are as follows:

[0025]

[0026] Among them, C and It consists of the ground-based actual building mask and the predicted building mask; Cd and Represents pixels within d distance of the building boundary;

[0027] S36. During the algorithm execution, the scale difference is specifically manifested as: small, medium, and large; the size corresponds to pixels between 32×32, 32×32 and 96×96, and larger than 96×96.

[0028] Preferably, step S4 specifically includes the following steps:

[0029] S41. A visual comparison of PolyWorld, HiSup, and the generated extraction results was performed from top to bottom.

[0030] S42. The visualization comparison results show that the performance difference in extracting medium and large building bodies is limited; the last column of the visualization shows that its left side outperforms HiSup in terms of body extraction performance.

[0031] S43. The quantitative evaluation results applied to the AICrowd building dataset achieved an AP of 68.5%, which is competitive with other methods.

[0032] S44. Using HiSup as a benchmark, test the algorithm's performance on automatic mixed precision, DST, high-resolution networks, high spatial resolution supervised targets, and additional semantic segmentation branches.

[0033] Compared with existing technologies, the beneficial effects of this invention are as follows: Dynamic scaling training (DST) is applied during the model training phase to mitigate the negative impact of scale differences on performance; a scale-aware backbone, i.e., a high-resolution network, is employed to enhance feature representation; finally, high-resolution supervision targets are used to further improve performance; extensive comparative studies on the AICrowd building dataset show that our method is highly competitive, with an average accuracy of 68.5%; on the Inria building dataset, this application surpasses previous state-of-the-art methods with an Intersection of Union (IoU) of 82.6% and an accuracy of 97.4%; this application achieves significantly higher accuracy than HiSup in AP, AP-small (APs), Average Recall (AR), and AR-small (ARs) metrics, with 38.4%, 26.8%, 46.1%, and 34.0% respectively; and this application demonstrates high performance in building boundary delimitation. Attached Figure Description

[0034] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0035] Figure 1 This invention uses HigherHRNet for building boundary extraction. (Network structure diagram)

[0036] Figure 2 A schematic diagram of the network structure using the target extraction branch in this invention;

[0037] Figure 3 Building boundary extraction results from high-resolution remote sensing images of the AICrowd building dataset using PolyWorld, HiSup, and HigherHRNet (corresponding to rows one, two, and three respectively);

[0038] Figure 4 Building boundary extraction results from high-resolution remote sensing images of the Inria building dataset using HiSup and HigherHRNet (corresponding to rows one and two respectively);

[0039] Figure 5 Building boundary extraction results from high-resolution remote sensing images of the WHU Wuhan University building dataset using PolyWorld, HiSup, and HigherHRNet (corresponding to rows one, two, and three respectively).

[0040] Figure 6 Building boundary extraction results from WBD Waterloo Buildings dataset high-resolution remote sensing image (0.12m / pixel) using HiSup and HigherHRNet (corresponding to rows one and two respectively);

[0041] Figure 7 Building boundary extraction results from WBD Waterloo Building Dataset high-resolution remote sensing image (0.3m / pixel) using PolyWorld, HiSup, and HigherHRNet (corresponding to rows one, two, and three respectively). Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to represent selected embodiments of the invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0043] Please see Figure 1-7 In this embodiment of the invention, a method for automatically extracting building boundaries based on high-resolution remote sensing imagery includes the following steps:

[0044] S1. Filter the relevant building dataset;

[0045] S2. Use a high-resolution network trained with dynamic scaling to extract building boundaries from optical remote sensing images;

[0046] S3. Use relevant evaluation parameters to quantitatively define the extraction accuracy;

[0047] S4. Compare and evaluate the method's performance by evaluating the differences between the evaluation parameters and the dataset.

[0048] Preferably, step S1 specifically includes the following steps:

[0049] S11. Select four public building datasets to broadly evaluate the performance of the inventive network and test its robustness; the datasets should meet the requirement of being widely used for building outline extraction.

[0050] S12. The filtered datasets include the AICrowd architecture dataset, the Inria architecture dataset, the WHU architecture dataset, and the Waterloo architecture dataset.

[0051] S13. The imagery in the dataset includes three bands: red, green, and blue, with different spatial resolutions of 0.3m, 0.075m, and 0.12m, respectively, and the geographical coverage areas vary.

[0052] Preferably, step S2 specifically includes the following steps:

[0053] S21. A novel HiSup-based method was developed to mitigate the scale difference problem: Mask-and-Vertices Attraction used in HiSup was applied; predicted vertices and masks were used as input to initialize polygons; local nonmaximum suppression was applied to sparse vertices; with the help of predicted offset vectors, refined vertices were used to simplify the initialized polygons, removing redundant and low-confidence vertices from the initialized polygons; if adjacent edges in each polygon are nearly parallel, they are further merged.

[0054] S22. A Dynamic Scale Training (DST) strategy was added to the model optimization. DST overcomes scaling differences by applying tiling images and supervised objectives. The supervised objectives are guided by dynamic feedback. The feedback is the proportion of the loss of small objects to all objects.

[0055] S23. The original high-resolution network (HRNet)v2 was replaced with a scale-aware high-resolution network. HRNet showed excellent performance in feature extraction and representation using multi-level features, with repeated information exchange at each stage. However, in the final stage of HRNet, the highest resolution of the features was 1 / 4 of the input value. Information loss and scale difference suppressed the performance of HRNet. HigherHRNet was proposed by adding a scale-aware module to HRNet. The scale-aware module mainly consists of a deconvolution module and 4 residual blocks. Two features with different spatial resolutions are downsampled and concatenated into 128×128.

[0056] Preferably, step S3 specifically includes the following steps:

[0057] S31. Input step S1 to filter high-resolution remote sensing images of the building dataset;

[0058] S32. Use the scale-aware high-resolution network from step S2 to train the image dataset and extract the building outlines from the images.

[0059] S33. Use rank-level evaluation metrics for instance segmentation and object detection in computer vision and remote sensing applications, including mean precision (AP), mean recall (AR), AP50, and AP75.

[0060] S34. In addition to conventional evaluation metrics, AP-Small (APs), AP-Medium (APm), AP-Large (APL), AR-Small (ARs), AR-Medium (ARm), and AR-Large (ARL) were also used as evaluation metrics to assess the extraction results and algorithm performance.

[0061] S35. A constraint metric, APboundary, has been added as an evaluation metric during the internal evaluation process. APboundary is the average accuracy calculated based on the boundary intersection unit (IoU). The specific steps for solving the boundary IoU are as follows:

[0062]

[0063] Among them, C and It consists of the ground-based actual building mask and the predicted building mask; Cd and Represents pixels within d distance of the building boundary;

[0064] S36. During the algorithm execution, the scale difference is specifically manifested as: small, medium, and large; the size corresponds to pixels between 32×32, 32×32 and 96×96, and larger than 96×96.

[0065] Preferably, step S4 specifically includes the following steps:

[0066] S41. A visual comparison of PolyWorld, HiSup, and the generated extraction results was performed from top to bottom.

[0067] S42. The visualization comparison results show that the performance difference in extracting medium and large building bodies is limited; the last column of the visualization shows that its left side outperforms HiSup in terms of body extraction performance.

[0068] S43. The quantitative evaluation results applied to the AICrowd building dataset achieved an AP of 68.5%, which is competitive with other methods.

[0069] S44. Using HiSup as a benchmark, test the algorithm's performance on automatic mixed precision, DST, high-resolution networks, high spatial resolution supervised targets, and additional semantic segmentation branches.

[0070] An automatic building boundary extraction method based on high-resolution remote sensing imagery includes the following steps:

[0071] S1. Select relevant building datasets that can be effectively applied to building boundary extraction, and choose the datasets for the application of the method in the invention to extract building boundaries. Figures 3-7 That is, the extraction results displayed on the selected dataset;

[0072] S11. Select four public building datasets: AICrowd Building Dataset, Inria Building Dataset, Wuhan University Building Dataset (WHU), and Waterloo Area Building Dataset (WBD).

[0073] S2. Establish a high-resolution network with dynamic scale training (HigherNet-DST). By applying dynamic scale training, using a scale-aware high-resolution network, and employing a high-resolution supervised target based on HiSup, the scaling discrepancy problem is mitigated, and the performance of building boundary delineation is improved (see details of the network structure). Figure 1 and Figure 2 );

[0074] S21. A novel method based on Mask-and-Vertices Attraction in HiSup was developed for dynamically scale-trained high-resolution networks (HigherNet-DST). This method uses predicted vertices and masks as input to initialize polygons, ultimately mitigating scale differences (see...). Figure 1 (Schematic diagram of HigherHRNet network architecture);

[0075] S22. Local Non-Maximum Suppression (NMS) is applied to sparse vertices. With the help of the predicted offset vector, the refined vertices are used to simplify the initialized polygon, removing redundant and low-confidence vertices from the initialized polygon;

[0076] S23. A Dynamic Scale Training (DST) strategy was incorporated into the model optimization. This strategy overcomes scale differences by applying collaged images and supervised objectives, with the supervised objectives guided by dynamic feedback (see...). Figure 2 (Diagram of the network structure of the target extraction branch);

[0077] S24. The superior performance of multi-level feature extraction and representation used in the original high-resolution network (HRNet) v2 was replaced by the scale-aware HigherHRNet.

[0078] S25. Add a scale-aware module to HigherHRNet to reduce the impact of information loss and scale differences on HRNet performance.

[0079] S26. Downsample two features with different spatial resolutions and concatenate them into a 128×128 array to save on excessive computational cost.

[0080] S3. Use the HigherNet-DST network to extract building boundaries, and evaluate the accuracy of building boundary extraction using relevant evaluation parameters (see details). Figures 3-7 );

[0081] S31. The high-resolution network (HigherNet-DST) of the invention is used to train the high-resolution remote sensing images of the four selected building datasets respectively, and the building outlines in the images are extracted.

[0082] S34. The differences in building scale in the image are specifically manifested as follows: small, medium, and large sizes correspond to 32×32 pixels, pixels between 32×32 and 96×96 pixels, and pixels larger than 96×96 pixels, respectively.

[0083] S32. Using average precision (AP), average recall (AR), AP50, AP75, AP-Small (APs), AP-Medium (APm), AP-Large (APL), AR-Small (ARs), AR-Medium (ARm), and AR-Large (ARL), which are widely used in instance segmentation and object detection in computer vision and remote sensing applications, the performance of building contour extraction is quantitatively evaluated.

[0084] S33. A limiting indicator, APboundary, has been added as an evaluation indicator during the internal evaluation process. APboundary is the average accuracy calculated based on the boundary intersection (IoU). The specific steps for solving the boundary IoU are as follows:

[0085]

[0086] Among them, C and These are the actual building masks and the predicted building masks. Cd and This represents pixels within a distance d from the building boundary.

[0087] S4. Compare and evaluate the method's performance by comparing the evaluation parameters and the differences in the dataset;

[0088] S41. The extraction results of PolyWorld, HiSup, and the method of this invention on different datasets were visually compared. The visualization results show differences in performance in extracting small, medium, and large buildings. Figures 3-7 );

[0089] The extraction results for large buildings on the S42 and AICrowd building datasets did not show significant changes due to the method change, but the extraction results for small buildings showed greater differences. In most cases, the extraction performance of the method in this invention was better than PolyWorld and HiSup, and only slightly worse than HiSup in a few cases. Ultimately, the quantitative evaluation results of this invention on the AICrowd building dataset achieved an AP of 68.5%, which is competitive with other methods. Figure 3 );

[0090] The extraction results on the S43 and Inria building datasets show that for large, medium, and small buildings, the invented method HigherNet-DST outperforms HiSup, achieving 38.4% higher AP than HiSup (9.4%), an increase of over 20% in ARL, and increases of over 10% in AP50, AP75, APm, AR, ARs, and ARm. Figure 4 );

[0091] On the S44 and WHU (Wuhan University of Technology) architectural datasets, the method of this invention outperforms HiSup and PolyWorld, especially in the extraction of small building boundaries. Figure 5 );

[0092] On the S45 and WBD Waterloo building datasets, HiSup and PolyWorldbia perform similarly, while the method of this invention performs better in distinguishing between building roofs and their side walls, as well as small buildings. Figure 6 , Figure 7 ).

[0093] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for automatic building boundary extraction based on high-resolution remote sensing imagery, characterized in that: Includes the following steps: S1. Filter the relevant building dataset; S2. Use a high-resolution network trained with dynamic scaling to extract building boundaries from optical remote sensing images; S3. Use relevant evaluation parameters to quantitatively define the extraction accuracy; S4. Compare and evaluate the method's performance by comparing the evaluation parameters and the differences in the dataset; S1 specifically includes the following steps: S11. Select four public building datasets to broadly evaluate the performance of the inventive network and test its robustness; the datasets should meet the requirement of being widely used for building outline extraction. S12. The filtered datasets include the AICrowd architecture dataset, the Inria architecture dataset, the WHU architecture dataset, and the Waterloo architecture dataset. S13. The images in the dataset include three bands: red, green, and blue, with different spatial resolutions of 0.3m, 0.075m, and 0.12m, respectively, and the geographical coverage areas vary. S2 specifically includes the following steps: S21. A novel HiSup-based method was developed to mitigate the scale difference problem: Mask-and-Vertices Attraction used in HiSup was applied; predicted vertices and masks were used as input to initialize polygons; local nonmaximum suppression was applied to sparse vertices; with the help of predicted offset vectors, refined vertices were used to simplify the initialized polygons, removing redundant and low-confidence vertices from the initialized polygons; if adjacent edges in each polygon are nearly parallel, they are further merged. S22. A dynamic scaling training strategy was added to the model optimization. Dynamic scaling training overcomes scaling differences by applying collage images and supervised objectives. The supervised objectives are guided by dynamic feedback. The feedback is the proportion of the loss of small objects to all objects. S23. The original high-resolution network v2 was replaced with a scale-aware high-resolution network.

2. The method for automatic building boundary extraction based on high-resolution remote sensing imagery according to claim 1, characterized in that: S3 specifically includes the following steps: S31. Input step S1 to filter high-resolution remote sensing images of the building dataset; S32. Use the scale-aware high-resolution network from step S2 to train the image dataset and extract the building outlines from the images. S33. Use rank-level evaluation metrics for instance segmentation and object detection in computer vision and remote sensing applications, including mean precision, mean recall, AP50, and AP75. S34. In addition to conventional evaluation metrics, AP-Small, AP-Medium, AP-Large, AR-Small, AR-Medium, and AR-Large were also used as evaluation metrics to assess the extraction results and algorithm performance. S35. A limiting indicator, APboundary, has been added as an evaluation metric during the evaluation process. APboundary is the average accuracy based on the joint calculation of boundary intersections. The specific steps for solving the boundary IoU are as follows: Among them, C and It consists of the ground-based actual building mask and the predicted building mask; Cd and Represents pixels within d distance of the building boundary; S36. During the algorithm execution, the scale difference is specifically manifested as: small, medium, and large; the size corresponds to pixels between 32×32, 32×32 and 96×96, and larger than 96×96.

3. The method for automatic building boundary extraction based on high-resolution remote sensing imagery according to claim 1, characterized in that: S4 specifically includes the following steps: S41. A visual comparison of PolyWorld, HiSup, and the generated extraction results was performed from top to bottom. S42. The visualization comparison results show that the performance difference in extracting medium and large building bodies is limited; the last column of the visualization shows that its left side outperforms HiSup in terms of body extraction performance. S43. The quantitative evaluation results applied to the AICrowd building dataset achieved an AP of 68.5%, which is competitive with other methods. S44. Using HiSup as a benchmark, test the algorithm's performance on automatic mixed precision, DST, high-resolution networks, high spatial resolution supervised targets, and additional semantic segmentation branches.

Images