A building instance segmentation and regularization method and system based on edge optimization

Abstract

The application relates to a kind of building instance segmentation and regularization method and system based on edge optimization, method includes the following steps: obtaining the building image data set of multiple scenes;Based on the building image data set, train the building positioning and extraction model based on edge buffer;The mask information of building in target building image is obtained by using the trained building positioning and extraction model, and the mask information of building is converted into regularized vector contour information by judging the main direction of outline.Compared with prior art, the application faces the current high-precision building automation vector mapping demand, and designs the building instance segmentation and regularization method based on edge buffer, which can effectively improve the building mask quality and realize the building map production of large scene.

Description

TECHNICAL FIELD

[0001] The present application relates to a building instance segmentation and regularization method and system based on edge optimization. BACKGROUND

[0002] Buildings are the main places for human life and production. Single building extraction and polygonal building extraction based on high-resolution remote sensing images are helpful for understanding the urbanization process, assessing urban construction risks, and timely updating the urban geographic information database, which is of great significance for optimizing urban spatial layout and building a sustainable digital smart city. High-resolution aerial remote sensing images can provide rich ground feature detail information, providing basic data for obtaining high-precision building vector polygons. However, it is still a time-consuming and labor-intensive task to convert high-resolution remote sensing images into building vector maps with rich details and accurate positioning. Obviously, automatic extraction and vectorization of buildings based on high-resolution aerial images can greatly reduce the cost of manpower and time. After decades of exploration, this field has accumulated rich research results, but due to the diversity of building structures, the complexity of urban environment, and the difference of remote sensing imaging, high-precision building vector mapping based on remote sensing images is still a challenging task. At present, the building contour information used in the fields of geographic information science and surveying and mapping engineering mainly depends on manual annotation, which consumes a lot of manpower and material resources.

[0003] Traditional methods generally start from the geometric characteristics of building regularization, combine the spectral characteristics of roofs and shadows, and design a feature expression suitable for buildings to realize automatic extraction and recognition of buildings. Limited by season, light, angle, sensor type and other differences, traditional building extraction methods designed relying on specific experience are easily disturbed by noise and other ground objects, and do not have wide applicability, so they are gradually replaced by artificial intelligence algorithms represented by deep learning.

[0004] There are problems of adhesion of adjacent buildings and unclear building edge contour in the current building extraction in the field of remote sensing, and there is currently a lack of an instance segmentation and regularization method to solve or partially solve the aforementioned problems. SUMMARY

[0005] The purpose of the present application is to overcome the defects of the prior art and provide a building instance segmentation and regularization method and system based on edge optimization to solve or partially solve the problems of adhesion of adjacent buildings and unclear building edge contour.

[0006] The purpose of the present application can be achieved by the following technical solutions:

[0007] In one aspect of the present application, a building instance segmentation and regularization method based on edge optimization is provided for effectively locating and distinguishing adjacent buildings and extracting building vector contours with rich edge details and regularized geometric characteristics, the method comprising the following steps:

[0008] Obtaining a multi-scene building image dataset;

[0009] Training a building location and extraction model based on edge buffer based on the building image dataset;

[0010] Using the trained building location and extraction model to predict the mask information of the building in the target building image, and converting the mask information of the building into regularized vector contour information by judging the main direction of the contour.

[0011] As a preferred technical solution, the building location and extraction model is configured to:

[0012] Feature extraction and propagation:

[0013] According to the input building image data, multi-scale features are extracted, global semantic segmentation features are obtained by multi-scale feature fusion through deformable convolution, and instance features are obtained by combining spatial and channel attention mechanisms for feature fusion and propagation in the region of interest, and semantic segmentation prediction and instance segmentation prediction are obtained based on the global semantic segmentation features and the instance features;

[0014] Building edge optimization:

[0015] Based on the global semantic segmentation features, instance features, semantic segmentation prediction and instance segmentation prediction, fusion features are obtained based on deformable convolution;

[0016] Based on the fusion features, the instance features for the next building edge optimization are obtained by upsampling;

[0017] Based on the prediction corresponding to the fusion features, the building edge is extracted by the Laplace operator, the uncertainty of the midpoint in the building edge buffer is calculated, and the predicted results are corrected using a multilayer perceptron with a preset number of points with high uncertainty as input, and the corrected prediction is used as the instance segmentation prediction for the next building edge optimization;

[0018] Based on the results of multiple building edge optimizations, the mask information of the building is obtained.

[0019] As a preferred technical solution, the global semantic segmentation features obtained by multi-scale feature fusion through deformable convolution are implemented based on the following formula:

[0020] F i = fa (D i +S i ,f Δ (D i +S i ))

[0021] wherein, F i represents the i-th layer feature, D i represents the deep layer feature after up-sampling, S i represents the shallow layer feature, f Δ is used to calculate the feature error, f a is used for feature alignment.

[0022] As a preferred technical solution, the joint space and channel attention mechanism are used for region of interest feature fusion and propagation to obtain instance features based on the following formula:

[0023]

[0024] wherein, and respectively represent the i-th and i-1-th layer region of interest features, f sigmoid represents the Sigmoid normalization process, f de represents the feature dimension reduction process, and f se represents the calculation process of the channel attention mechanism.

[0025] As a preferred technical solution, the fusion features obtained by merging and deformable convolution are based on the following formula:

[0026]

[0027] wherein, are instance segmentation features, instance segmentation predictions, semantic segmentation features, and semantic segmentation predictions, respectively, is the i-1-th building edge optimized fusion feature, are the merged instance segmentation features and predictions, and the merged semantic segmentation features and predictions, respectively, are are the features after reducing the number of channels, concat() represents channel splicing, FAM() represents feature fusion based on deformable convolution, Upsample() represents up-sampling, and Predictor() represents a prediction operator.

[0028] As a preferred technical solution, the process of converting the mask information of the building into vector contour information includes:

[0029] Based on the mask information, the building contour is extracted and the edge points are simplified to obtain a simplified edge contour curve S1;

[0030] Calculate the length of all line segments in the edge contour curve S1 that do not coincide with the rectangular frame, and take the direction of the longest line segment as the main direction;

[0031] Calculate the angle between each line segment and the main direction, and rotate each line segment to be parallel or perpendicular to the main direction according to the angle, to obtain the adjusted edge contour curve S2;

[0032] Based on the adjusted edge contour curve S2, adjust the length of the line segment according to the distance between adjacent line segments to make the contour curve closed, and obtain the vector contour information.

[0033] As a preferred technical solution, the process of obtaining a multi-scene building image data set comprises:

[0034] Obtain multi-scene building image data and corresponding label information;

[0035] Crop the building image data, convert the label information into a preset format, and realize the correspondence of image name, building positioning and contour;

[0036] Regularize for training influence;

[0037] Construct a multi-scene building image data set.

[0038] As a preferred technical solution, the process of obtaining a multi-scene building image data set comprises:

[0039] Obtain the scene image to be detected to obtain multiple image blocks in a partially overlapping manner;

[0040] Take the image block as the input of the trained building positioning and extraction model to predict building classification information and mask information;

[0041] Based on the corresponding building classification information of the image block, average the building probability of the overlapping area and merge the same buildings distributed in different image blocks;

[0042] For each merged building, convert the building mask information into a regularized vector contour,

[0043] Wherein, averaging the building probability of the overlapping area and merging the same buildings distributed in different image blocks comprises:

[0044] Construct a matrix matching the size of the scene image to be detected, determine whether there is an overlapping area at the corresponding position of the matrix for each building object of the image block, merge the building objects with an intersection-over-union greater than a preset value, and assign a new number to the building objects with an intersection-over-union less than the preset value.

[0045] When the building object of the image block overlaps with multiple objects on the matrix, the multiple objects and the building object of the image block are merged into one object, and the minimum value of the number is taken as the number of the merged object.

[0046] In another aspect of the present application, a building instance segmentation and regularization system based on edge optimization is provided, comprising:

[0047] A data set construction module is configured to obtain a plurality of scene building image data sets.

[0048] A model construction module is configured to construct and train a building positioning and extraction model based on the edge buffer based on the building image data set.

[0049] A contour regularization processing module is configured to predict the mask information of the building in the target building image using the trained building positioning and extraction model, and convert the mask information of the building into regularized vector contour information by judging the main direction of the contour.

[0050] As a preferred technical solution, the model construction module comprises:

[0051] A feature extraction network is configured to extract multi-scale features from the input building image data, perform multi-scale feature fusion through deformable convolution to obtain global semantic segmentation features, perform feature fusion and propagation in the region of interest through joint spatial and channel attention mechanisms to obtain instance features, and obtain semantic segmentation prediction and instance segmentation prediction based on the global semantic segmentation features and the instance features.

[0052] A building edge optimization unit is configured to obtain fusion features through merging and deformable convolution based on the global semantic segmentation features, the instance features, the semantic segmentation prediction, and the instance segmentation prediction, obtain instance features for the next building edge optimization through upsampling based on the fusion features, extract building edges through a Laplacian operator based on the corresponding prediction of the fusion features, calculate the uncertainty of the midpoint in the building edge buffer, use the preset number of points with high uncertainty as input, and correct the prediction result using a multilayer perceptron, wherein the corrected prediction is used as the instance segmentation prediction for the next building edge optimization, and the mask information of the building is obtained based on the results after multiple building edge optimizations.

[0053] Compared with the prior art, the present application has at least one of the following beneficial effects:

[0054] (1) Improve the extraction effect of the building edge: the present application uses the trained building positioning and extraction model to predict the mask information of the building in the target building image, and converts the mask information of the building into regularized vector contour information by judging the main direction of the contour, which can effectively improve the edge characteristics of the building extraction result, and can achieve good visual effect under different vectorization post-processing methods.

[0055] (2) Small feature loss: the present application designs multi-scale feature fusion and propagation, effectively preserves and propagates feature information of different scales, and reduces the loss of feature details caused by traditional feature fusion and propagation models.

[0056] (3) Suitable for large scene building map making: the present application obtains multiple image blocks in a partially overlapping manner as the input of the trained building positioning and extraction model, predicts building classification information and mask information, averages the building probability of the overlapping area based on the corresponding building classification information of the image blocks, and merges the same buildings distributed in different image blocks, which can effectively improve the building mask quality and realize large scene building map making.

[0057] (4) Retain edge details: the present application designs a secondary optimization process based on edge buffer zone to solve the problem of poor edge details of building mask, fully considers the structural characteristics of building edge in point, line and surface, and effectively retains the edge details of building.

[0058] (5) Improve the visual effect: the present application corrects the wrong main direction caused by the building positioning frame, which effectively improves the visual effect of the building vector result. BRIEF DESCRIPTION OF DRAWINGS

[0059] Figure 1 The flow chart of the building instance segmentation and regularization method based on edge optimization in the embodiment;

[0060] Figure 2 The architecture schematic diagram of the building instance segmentation and regularization method based on edge buffer zone in the embodiment;

[0061] Figure 3 The schematic diagram of the feature fusion and propagation process in the embodiment;

[0062] Figure 4 The edge optimization framework schematic diagram in the embodiment;

[0063] Figure 5 The schematic diagram of the building instance segmentation and regularization system based on edge optimization in the embodiment;

[0064] Figure 6 An illustration of a feature fusion process based on deformable convolution in an embodiment;

[0065] Figure 7 An illustration of an electronic device in an embodiment. DETAILED DESCRIPTION

[0066] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative effort should fall within the protection scope of the present application.

[0067] Embodiment 1

[0068] In view of the problems in the prior art described above, the present embodiment provides a building instance segmentation and regularization method based on an edge buffer area. As shown in a framework diagram of the method, Figure 2 the method provides a Poly-BRBLE framework combining a single building location and extraction model (BRBLE) based on an edge buffer area and a contour regularization post-processing method, and realizes a complete process from high spatial resolution aerial images to building vector base maps.

[0069] Referring to Figure 1 , the method includes the following steps:

[0070] Step S1, constructing a high-resolution building image dataset of different scenes.

[0071] The collected high-resolution remote sensing images are cropped, and the corresponding labels are converted into json format, realizing one-to-one correspondence of image names, building location and contour. The training images are regularized to control the distribution of data and ensure the stability of training.

[0072] Step 2, inputting the images and labels in the training set into the single building location and extraction model BRBLE based on the edge buffer area for training to obtain the weight information required for model inference. The structure of the BRBLE model will be described below from two aspects of feature extraction and propagation and building edge optimization, including steps S201-S202:

[0073] Step S201, feature extraction and propagation process.

[0074] A backbone feature extraction network (such as Resnet101, Resnet50, etc.) is used for feature extraction, and an improved feature fusion and propagation structure is used to complete multi-scale feature fusion and cross-layer feature fusion and transmission of the region of interest. The specific framework diagram is as follows: Figure 3The deformable convolution is introduced in the multi-scale feature fusion process to reduce the inconsistency between the multi-scale feature layers. The calculation process is as follows:

[0075] F i = f a (D i +S i , f Δ (D i +S i ))

[0076] Where F i represents the i-th layer feature, D i represents the up-sampled deep feature, S i represents the shallow feature, f Δ is used to calculate the feature error, and f a is used for feature alignment.

[0077] The traditional region of interest feature fusion and propagation process will obtain the corresponding feature layer number according to the proposal box scale. Since this method ignores other feature layers, the feature details are lost. To solve this problem, the embodiment designs a feature fusion and propagation process combined with spatial and channel attention mechanisms:

[0078]

[0079] Where, and represent the i-th and i-1-th region of interest features, f sigmoid represents the Sigmoid normalization process, f de represents the feature dimension reduction process, and f se represents the calculation process of the channel attention mechanism.

[0080] Step S202, building edge optimization process.

[0081] In order to improve the problem of fuzzy building edge after instance segmentation, a building edge optimization process is proposed, and the process framework is as Figure 4 shown. Through the newly added semantic segmentation head, more detailed global semantic segmentation information is obtained, which provides more semantic details for the mask up-sampling process of low detail information. Therefore, the instance feature and semantic feature fusion module SAIU containing the up-sampling process is provided.

[0082] Take the instance feature with scale (B, C, H, W) as an example, the corresponding cropped semantic feature scale is (B, C', H, W). In order to limit the parameter amount of the model, the instance feature fused by SAIU has higher spatial resolution and lower channel number, so C and C' are not equal in the upsampling process (not including the initial state), and a 1*1 convolution needs to be added to the semantic feature layer to align the channel number.

[0083] The input of the SAIU module contains four parts, which are instance segmentation feature Instance segmentation prediction Semantic segmentation feature Semantic segmentation prediction Among them, the instance segmentation feature and prediction are merged into The semantic segmentation feature and prediction are merged into The channel numbers of and are reduced respectively, denoted as and And introduce deformable convolution to adjust the spatial information, reduce the influence of spatial misalignment on and fusion effect. The fusion result is denoted as The upsampled feature is into the next SAIU module.

[0084]

[0085] Among them, FAM() represents the feature fusion process based on deformable convolution. The specific process is as follows:

[0086] Through convolution operation f o Get the spatial misalignment information Δ i-1 between and .

[0087]

[0088] For each pixel of , it is corrected by , where ω n represents the weight of the convolution kernel, represents the spatial misalignment information corresponding to ω n . Finally, the adjusted feature is added to .

[0089] The probability distribution of the prediction results of buildings is observed, and it is found that although buildings and non-buildings have obvious linear boundaries in images, they appear as a planar transition zone with gradually decreasing probability from buildings to non-buildings in the prediction result map, which is called an "edge buffer". Compared with a linear edge, the edge buffer can include more detailed information and environmental information, and focuses the attention of the model on the difficult-to-distinguish building boundaries. Obviously, the edge buffer can also be regarded as a set of suspected building points (p) whose probability distribution is around 0.5. Combined with the multi-layer perception (MLP) network and the shallow feature information corresponding to the p points, the probability value of the suspected points can be revised again to further improve the processing capability of the model for the edges of buildings.

[0090] Optimization of edge points (Edge Points) and buffer zone (Edge Buffer). The operation process is as follows: based on the Laplace operator, the building edge is extracted, the edge width is set to w, and the edge buffer zone is obtained. Then, the building edge buffer zone prediction result of i rounds is used to replace the building edge buffer zone of i-1 rounds, and the result is recorded as The uncertainty degree of is calculated. The K points with the highest uncertainty are input into the multi-layer perception (MLP) to revise the prediction result. The revised result is used as input into the SAIU, that is, into the next round of upsampling.

[0091] Step S3, after obtaining the mask of the building, an improved contour regularization algorithm is proposed based on the "building regularization algorithm based on the main direction" proposed by Wei et al., considering that the cutting of the detection frame on the building contour will affect the judgment of the "contour main direction". The following is the specific building regularization process, which specifically includes S301-S304.

[0092] Step S301, the OpenCV is used to extract the building contour, and the Douglas-Peucker algorithm (Douglas-Peucker Algorithm) is used to simplify the edge points. The simplified edge contour curve is recorded as S1.

[0093] Step S302, the lengths of all line segments in the edge contour curve S1 that do not coincide with the rectangular frame are calculated, and the direction of the longest side is recorded as the "main direction".

[0094] Step S303, the angles between the remaining all side lengths and the "main direction" are calculated, the line segments with angles within 45 degrees are regarded as "parallel to the main direction", and the line segments with angles within 45 to 90 degrees are regarded as "perpendicular to the main direction". The main direction line segment is kept unchanged, and the above line segments are rotated to be parallel or perpendicular to the main direction. The adjusted edge contour curve is recorded as S2.

[0095] Step S304, the adjusted contour curve S2 is difficult to close, so the line segment is extended or increased according to the distance between adjacent line segments. The adjusted edge contour curve is denoted as S3.

[0096] Step S4, the large scene building detection result is spliced.

[0097] Considering that directly predicting the building vector contour in a large scene image has very high requirements for computer computing power, storage space and other hardware devices, the embodiment adopts a processing method of “first cropping and then splicing”. In the image preprocessing process, a certain overlap interval is set to crop the building image of the large scene into N×N size image blocks, then the building positioning and extraction on the image blocks are realized through step S2, and finally the building mask after splicing screening is converted into a vector contour through step S3. The splicing process of the building image block will be described below:

[0098] The building instance segmentation result of the processing image block (i.e. the instance segmentation result of the BRBLE model) takes the average value of the building probability of the overlapping area and merges the same buildings distributed on different image blocks. The merging process is: a matrix Z o with the same size as the large scene image is newly created to store the building instance segmentation data after splicing. Each building object is looped to judge whether the object has an overlapping area on Z o , the building objects with IoU greater than ω are merged, and the building objects with IoU less than ω are assigned with new numbers. When a building object overlaps with multiple objects on Z o , all the involved objects are merged into one object, and the minimum value of their maximum numbers is calculated as the number of the merged building object.

[0099] Each building on Z o is looped to obtain the building regularized contour under the large scene through step S3.

[0100] In summary, the method has the following characteristics:

[0101] (1) A building instance segmentation and regularization method based on edge buffer is designed to meet the current demand for high-precision automatic building vector mapping, which can effectively improve the building mask quality and realize the production of building map in large scenes.

[0102] (2) A multi-scale feature fusion and propagation module is designed to effectively save and propagate feature information of different scales, reducing the loss of feature details caused by traditional feature fusion and propagation models.

[0103] (3) In view of the problem of poor edge detail of the current building mask, a secondary optimization process based on edge buffer is designed, fully considering the structural characteristics of the building edge in points, lines and surfaces, and effectively preserving the edge details of the building.

[0104] (4) An improvement is proposed for the current mainstream "building regularization algorithm based on main direction", which corrects the wrong main direction caused by the building positioning frame, effectively improving the visualization effect of the building vector result.

[0105] Embodiment 2

[0106] Based on embodiment 1, referring to Figure 5 , the embodiment provides a building instance segmentation and regularization system based on edge optimization, which is used to realize the building instance segmentation and regularization method of embodiment 1, comprising:

[0107] (1) A data set construction module is used to obtain a multi-scene building image data set;

[0108] (2) A model construction module is used to construct and train a building positioning and extraction model based on edge buffer based on the building image data set;

[0109] (3) A contour regularization processing module is used to predict the mask information of the building in the target building image by using the trained building positioning and extraction model, and convert the mask information of the building into regularized vector contour information by judging the contour main direction.

[0110] Preferably, the model construction module comprises:

[0111] A feature extraction network is used to extract multi-scale features according to the input building image data, obtain global semantic segmentation features by multi-scale feature fusion through deformable convolution, fuse and propagate the features of the region of interest through joint spatial and channel attention mechanism, obtain instance features, and obtain semantic segmentation prediction and instance segmentation prediction based on global semantic segmentation features and instance features.

[0112] A building edge optimization unit is used to obtain fusion features by merging and deformable convolution based on global semantic segmentation features, instance features, semantic segmentation prediction and instance segmentation prediction, obtain instance features for the next building edge optimization by upsampling based on the fusion features, extract the building edge by Laplace operator based on the prediction corresponding to the fusion features, calculate the uncertainty of the points in the building edge buffer, and use the multi-layer perception to correct the prediction result by using the preset number of points with high uncertainty as input. The corrected prediction is used as the instance segmentation prediction for the next building edge optimization, and the mask information of the building is obtained based on the results of multiple building edge optimizations.

[0113] The system has the following characteristics:

[0114] (1) The building instance segmentation model based on the edge buffer can effectively improve the edge features of the building extraction results, and good visual effects can be achieved under different vectorization post-processing methods.

[0115] (2) The feature fusion and propagation process and the edge optimization process can be applied to different instance segmentation models and have strong migration.

[0116] (3) The building instance segmentation and regularization method based on the edge buffer can realize large-scene building vector map production and has a wide range of application scenarios.

[0117] Embodiment 3

[0118] The embodiment provides an electronic device, including one or more processors and memories, the memories have one or more programs stored therein, and the one or more programs include instructions for executing the building instance segmentation and regularization method as described in embodiment 1.

[0119] As Figure 7 described, at the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and of course, other hardware required by the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs to implement the building instance segmentation and regularization method described above. Figure 1 Of course, in addition to the software implementation, the present application does not exclude other implementation manners, such as logic devices or a combination of software and hardware, that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or a logic device.

[0120] The memory can include non-permanent memory in the computer readable medium, random access memory (RAM) and / or non-volatile memory such as read-only memory (ROM) or flash memory (flash RAM). The memory is an example of a computer readable medium.

[0121] Computer-readable media includes permanent and non-permanent, movable and non-movable media that can implement information storage by any method or technology. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible by a computing device. According to the definition herein, computer-readable media does not include transitory media such as modulated data signals and carriers.

[0122] The above description is merely a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present application, and these modifications or replacements should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for building instance segmentation and regularization based on edge optimization, characterized in that, The method comprises the following steps: Obtain a multi-scene building image dataset; Train an edge buffer-based building positioning and extraction model based on the building image dataset; Use the trained building positioning and extraction model to predict the mask information of the building in the target building image, and convert the mask information of the building into regularized vector contour information by judging the main direction of the contour, The building positioning and extraction model is configured to: Feature extraction and propagation: Extract multi-scale features from the input building image data, fuse the multi-scale features through deformable convolution to obtain global semantic segmentation features, fuse and propagate the features of the region of interest through a spatial and channel attention mechanism to obtain instance features, and obtain semantic segmentation prediction and instance segmentation prediction based on the global semantic segmentation features and the instance features; Building edge optimization: Based on the global semantic segmentation features, instance features, semantic segmentation prediction and instance segmentation prediction, obtain fusion features based on deformable convolution; Based on the fusion features, obtain instance features for the next building edge optimization through upsampling; Based on the prediction corresponding to the fusion feature, building edge is extracted by Laplacian operator, edge width is set, edge buffer is obtained, and the building edge buffer prediction result of the round is replaced the building edge buffer of the round, the result is recorded as , the uncertainty degree of is calculated, the point with the highest uncertainty is input into the multilayer perception machine MLP, and the prediction result is secondarily corrected. The corrected result is used as the up-sampling of the next round; Based on the results of multiple building edge optimizations, obtain the mask information of the building, The fusion features are obtained through merging and deformable convolution, which is realized based on the following formula: wherein, , , , are respectively instance segmentation features, instance segmentation predictions, semantic segmentation features, semantic segmentation predictions, is the first building edge optimized fused feature, , are respectively merged instance segmentation features and predictions, merged semantic segmentation features and predictions, , are respectively , are features after reducing the number of channels, denotes channel concatenation, denotes feature fusion based on deformable convolution, denotes up-sampling, denotes a prediction operator.

2. The edge optimization based building instance segmentation and regularization method according to claim 1, wherein, The global semantic segmentation features obtained through multi-scale feature fusion by deformable convolution are realized based on the following formula: wherein, denotes the layer feature, denotes the deep layer feature after upsampling, denotes the shallow layer feature, is used to calculate the feature error, is used for feature alignment.

3. The edge optimization based building instance segmentation and regularization method of claim 1, wherein, The instance features obtained by fusing and propagating the features of the region of interest through a spatial and channel attention mechanism are realized based on the following formula: in, and Respectively represent and Layer region of interest features, Represents the Sigmoid normalization process, represents the feature dimensionality reduction process, Represents the calculation process of the channel attention mechanism.

4. The edge optimization based building instance segmentation and regularization method of claim 1, wherein, The process of converting the mask information of the building into vector contour information comprises: Based on the mask information, building contour is extracted and edge points are simplified to obtain a simplified edge contour curve ; Computing edge profile curves The direction of the longest line segment is taken as the main direction. calculating an angle between each line segment and the main direction, rotating each line segment to be parallel to the main direction or perpendicular to the main direction according to the angle, to obtain an adjusted edge profile curve ; based on the adjusted edge profile curve , the length of the line segment is adjusted according to the distance between adjacent line segments, the profile curve is closed, and vector profile information is obtained.

5. The edge optimization based building instance segmentation and regularization method of claim 1, wherein, The process of obtaining a multi-scene building image dataset comprises: Obtain multi-scene building image data and corresponding annotation information; Crop the building image data, convert the annotation information into a preset format, and realize the correspondence between image name, building positioning and contour; Regularize for training influence; Construct a multi-scene building image dataset.

6. The edge optimization based building instance segmentation and regularization method of claim 1, wherein, The process of predicting the mask information of the building in the target building image using the trained building positioning and extraction model comprises: Obtain a scene image to be detected to obtain multiple image blocks in a partially overlapping manner; Use the image blocks as input of the trained building positioning and extraction model to predict building classification information and mask information; Based on the corresponding building classification information of the image blocks, average the building probabilities of the overlapping regions and merge the same buildings distributed in different image blocks; For each merged building, convert the mask information of the building into regularized vector contour information, Wherein, averaging the building probabilities of the overlapping regions and merging the same buildings distributed in different image blocks comprises: Construct a matrix matching the size of the scene image to be detected, judge whether there is an overlapping region at the corresponding position of the matrix for each building object of the image block, merge the building objects with an intersection over union greater than a preset value, and assign a new number to the building objects with an intersection over union less than a preset value; When the building object of the image block overlaps with multiple objects on the matrix, the multiple objects and the building object of the image block are merged into one object, and the minimum value of the number is taken as the number of the merged object.

7. A system for building instance segmentation and regularization based on edge optimization, the system comprising: The system for implementing the edge-optimized building instance segmentation and regularization method according to any one of claims 1-6 comprises: a dataset construction module configured to obtain a multi-scene building image dataset; a model construction module configured to construct and train a building positioning and extraction model based on the building image dataset; a contour regularization processing module configured to predict the mask information of a building in a target building image by using the trained building positioning and extraction model, and convert the mask information of the building into regularized vector contour information by judging the main direction of the contour.

8. The edge optimization based building instance segmentation and regularization system of claim 7, wherein, The model construction module comprises: a feature extraction network configured to extract multi-scale features from input building image data, perform multi-scale feature fusion by deformable convolution to obtain global semantic segmentation features, perform feature fusion and propagation of a region of interest by combining spatial and channel attention mechanisms to obtain instance features, and obtain semantic segmentation prediction and instance segmentation prediction based on the global semantic segmentation features and the instance features; a building edge optimization unit configured to obtain fusion features by merging and deformable convolution based on the global semantic segmentation features, the instance features, the semantic segmentation prediction, and the instance segmentation prediction, obtain instance features for next building edge optimization by upsampling based on the fusion features, extract building edges by a Laplacian operator based on the corresponding prediction of the fusion features, calculate the uncertainty of the midpoint in the building edge buffer, take a preset number of points with high uncertainty as input, correct the prediction result by using a multilayer perceptron, take the corrected prediction as the instance segmentation prediction for next building edge optimization, and obtain the mask information of the building based on the results after multiple building edge optimizations.

Images