Building instance segmentation and regularization method and system based on edge optimization

Abstract

The invention relates to a building instance segmentation and regularization method and system based on edge optimization. The method comprises the following steps: acquiring a multi-scene building image data set; based on the building image data set, training a building positioning and extracting model based on an edge buffer area; and utilizing the trained building positioning and extraction model to predict and obtain the mask information of the building in the target building image, and converting the mask information of the building into regularized vector contour information by judging the main direction of the contour. Compared with the prior art, the building instance segmentation and regularization method based on the edge buffer area is designed for the current high-precision building automatic vector mapping requirement, the building mask quality can be effectively improved, and large-scene building map making is achieved.

Description

Technical Field

[0001] The present invention relates to, in particular to, a method and system for segmenting and regularizing building instances based on edge optimization. Background Art

[0002] Buildings are the main places for human life and production. The extraction and vectorization of single buildings (Polygonal building extraction) based on high-resolution remote sensing images helps to understand the urbanization process, assess the risks of urban construction and update the urban geographic information database in a timely manner. It is of great significance to optimize the urban spatial layout and build a sustainable digital smart city. High-resolution aerial remote sensing images can provide rich information on ground features and provide basic data for obtaining high-precision building vector polygons. However, converting high-resolution remote sensing images into building vector base maps with rich details and accurate positioning is still a time-consuming and labor-intensive task. Obviously, the automated extraction and vectorization of buildings based on high-resolution aerial images can greatly reduce the manpower and time costs. After decades of exploration, this field has accumulated rich research results, but due to the diversity of building structures, the complexity of urban environments and the differences in remote sensing imaging, high-precision building vector mapping based on remote sensing images is still a very challenging task. At present, the building outline information used in the fields of geographic information science, surveying and mapping engineering, etc. mainly relies on manual annotation, and repetitive work consumes a lot of manpower and material resources.

[0003] Traditional methods generally start from the regular geometric characteristics of buildings, combine the spectral characteristics of roofs, shadows, etc., design a feature expression suitable for buildings, and realize automatic extraction and recognition of buildings. Limited by differences in seasons, lighting, angles, sensor types, etc., traditional building extraction methods that rely on specific experience are easily interfered by noise and other ground objects and are not widely applicable, so they are gradually replaced by artificial intelligence algorithms represented by deep learning.

[0004] Currently, in the extraction of buildings in the remote sensing field, there are problems such as the adhesion of adjacent buildings and unclear edge contours of buildings. Currently, there is a lack of an instance segmentation and regularization method to solve or partially solve the above problems. Summary of the invention

[0005] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and provide a method and system for building instance segmentation and regularization based on edge optimization to solve or partially solve the problems of adhesion of adjacent buildings and unclear building edge contours.

[0006] The purpose of the present invention can be achieved by the following technical solutions:

[0007] In one aspect of the present invention, a method for segmenting and regularizing building instances based on edge optimization is provided, which is used to effectively locate and distinguish adjacent buildings and extract building vector contours with rich edge details and conforming to regularized geometric features. The method comprises the following steps:

[0008] Obtain multi-scene building image datasets;

[0009] Based on the building image dataset, training a building positioning and extraction model based on edge buffer;

[0010] The trained building location and extraction model is used to predict the mask information of the building in the target building image. The mask information of the building is converted into regularized vector contour information by judging the main direction of the contour.

[0011] As a preferred technical solution, the building positioning and extraction model is configured as follows:

[0012] Feature extraction and propagation:

[0013] According to the input building image data, multi-scale features are extracted, and the global semantic segmentation features are obtained by multi-scale feature fusion through deformable convolution. The feature fusion and propagation of the region of interest are performed by combining the spatial and channel attention mechanisms to obtain instance features. Based on the global semantic segmentation features and the instance features, semantic segmentation prediction and instance segmentation prediction are obtained respectively;

[0014] Building edge optimization:

[0015] Based on the global semantic segmentation features, instance features, semantic segmentation predictions and instance segmentation predictions, a fusion feature is obtained based on deformable convolution;

[0016] Based on the fused features, instance features for next building edge optimization are obtained by upsampling;

[0017] Based on the prediction corresponding to the fusion feature, the building edge is extracted by the Laplace operator, the uncertainty of the midpoint of the building edge buffer is calculated, a preset number of points with high uncertainty are used as input, and the prediction result is corrected by using a multi-layer perceptron, and the corrected prediction is used as the instance segmentation prediction for the next building edge optimization;

[0018] Based on the results of multiple building edge optimizations, the mask information of the building is obtained.

[0019] As a preferred technical solution, the multi-scale feature fusion through deformable convolution to obtain the global semantic segmentation feature is implemented based on the following formula:

[0020] F i =fa (D i +S i ,f Δ (D i +S i ))

[0021] Among them, F i represents the i-th layer feature, D i represents the deep features after upsampling, S i represents shallow features, f Δ Used to calculate the characteristic error, f a Used for feature alignment.

[0022] As a preferred technical solution, the joint space and channel attention mechanism performs feature fusion and propagation of the region of interest, and obtains the instance feature based on the following formula:

[0023]

[0024] in, and Respectively represent the features of the region of interest at the i-th and i-1-th layers, f sigmoid Represents the Sigmoid normalization process, f de represents the feature dimensionality reduction process, f se Represents the calculation process of the channel attention mechanism.

[0025] As a preferred technical solution, the fusion feature obtained by merging and deformable convolution is implemented based on the following formula:

[0026]

[0027] in, They are instance segmentation features, instance segmentation predictions, semantic segmentation features, and semantic segmentation predictions. is the fusion feature of the i-1th building edge optimization, They are the merged instance segmentation features and predictions, the merged semantic segmentation features and predictions, They are Features after reducing the number of channels, concat() represents channel concatenation, FAM() represents feature fusion based on deformable convolution, Upsample() represents upsampling, and Predictor() represents the prediction operator.

[0028] As a preferred technical solution, the process of converting the mask information of the building into the vector outline information includes:

[0029] Based on the mask information, extract the building outline and simplify the edge points to obtain a simplified edge outline curve S1;

[0030] Calculate the lengths of all line segments in the edge contour curve S1 that do not overlap with the rectangular frame, and take the direction of the longest line segment as the main direction;

[0031] Calculate the angle between each line segment and the main direction, and rotate each line segment to be parallel to or perpendicular to the main direction according to the angle, so as to obtain an adjusted edge contour curve S2;

[0032] Based on the adjusted edge contour curve S2, the line segment length is adjusted according to the distance between adjacent line segments to close the contour curve and obtain vector contour information.

[0033] As a preferred technical solution, the process of obtaining a multi-scene building image dataset includes:

[0034] Obtain building image data of multiple scenes and corresponding annotation information;

[0035] Cropping the building image data, converting the annotation information into a preset format, and achieving correspondence between the image name, building location and outline;

[0036] Regularize for training effects;

[0037] Construct a multi-scene building image dataset.

[0038] As a preferred technical solution, the mask information of the building in the target building image is predicted by using the trained building positioning and extraction model, including:

[0039] Acquire an image of a scene to be detected, and obtain a plurality of image blocks in a partially overlapping manner;

[0040] Using the image block as input to the trained building location and extraction model to predict building classification information and mask information;

[0041] Based on the corresponding building classification information of the image blocks, the building probabilities in the overlapping areas are averaged and the same buildings distributed in different image blocks are merged;

[0042] For each merged building, the mask information of the building is converted into a regularized vector outline.

[0043] The process of averaging the building probabilities in the overlapping areas and merging the same buildings distributed in different image blocks includes:

[0044] Constructing a matrix matching the size of the scene image to be detected, determining, for each building object in the image block, whether there is an overlapping area at the corresponding position of the matrix, merging building objects whose intersection-and-union ratio is greater than a preset value, and assigning new numbers to building objects whose intersection-and-union ratio is less than the preset value;

[0045] When the building object of the image block overlaps with multiple objects on the matrix, the multiple objects and the building object of the image block are merged into one object, and the minimum value of the serial number is used as the serial number of the merged object.

[0046] Another aspect of the present invention provides a building instance segmentation and regularization system based on edge optimization, comprising:

[0047] The dataset construction module is used to obtain building image datasets of multiple scenes;

[0048] A model building module, used for building and training a building positioning and extraction model based on an edge buffer zone based on the building image dataset;

[0049] The contour regularization processing module is used to use the trained building positioning and extraction model to predict the mask information of the building in the target building image, and convert the mask information of the building into regularized vector contour information by judging the main direction of the contour.

[0050] As a preferred technical solution, the model building module includes:

[0051] A feature extraction network is used to extract multi-scale features according to the input building image data, obtain global semantic segmentation features by multi-scale feature fusion through deformable convolution, fuse and propagate the features of the region of interest by combining the spatial and channel attention mechanisms, and obtain instance features, and obtain semantic segmentation prediction and instance segmentation prediction based on the global semantic segmentation features and the instance features, respectively;

[0052] The building edge optimization unit is used to obtain fused features through merging and deformable convolution based on the global semantic segmentation features, the instance features, the semantic segmentation predictions and the instance segmentation predictions; based on the fused features, obtain the instance features of the next building edge optimization through upsampling; based on the predictions corresponding to the fused features, extract the building edge through the Laplace operator, calculate the uncertainty of the midpoint of the building edge buffer, use a preset number of points with high uncertainty as input, use a multi-layer perceptron to correct the prediction results, use the corrected predictions as the instance segmentation predictions for the next building edge optimization, and obtain the mask information of the building based on the results of multiple building edge optimizations.

[0053] Compared with the prior art, the present invention has at least one of the following beneficial effects:

[0054] (1) Improving the extraction effect of building edges: The present invention uses the trained building positioning and extraction model to predict the mask information of the building in the target building image. By judging the main direction of the contour, the mask information of the building is converted into regularized vector contour information, which can effectively improve the edge features of the building extraction results and achieve better visual effects under different vectorization post-processing methods.

[0055] (2) Small feature loss: The present invention designs multi-scale feature fusion and propagation, which effectively preserves and propagates feature information of different scales, reducing the loss of feature details caused by traditional feature fusion and propagation models.

[0056] (3) Suitable for scenarios where large-scale building map production is required: The present invention obtains multiple image blocks in a partially overlapping manner as the input of the trained building positioning and extraction model, predicts building classification information and mask information, and takes the average of the building probabilities in the overlapping areas based on the corresponding building classification information of the image blocks and merges the same buildings distributed on different image blocks, which can effectively improve the quality of building masks and realize large-scale building map production.

[0057] (4) Preserving edge details: In order to solve the problem of poor edge details of current building masks, the present invention designs a secondary optimization process based on edge buffer, which fully considers the structural characteristics of building edges in terms of points, lines and surfaces, and effectively preserves the edge details of buildings.

[0058] (5) Improving visualization effect: The present invention corrects the erroneous main direction caused by the building positioning frame, effectively improving the visualization effect of the building vector result. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 It is a flowchart of the building instance segmentation and regularization method based on edge optimization in an embodiment;

[0060] Figure 2 Schematic diagram of the architecture of the building instance segmentation and regularization method based on edge buffer in the embodiment;

[0061] Figure 3 Schematic diagram of feature fusion and propagation process in the embodiment;

[0062] Figure 4 This is a schematic diagram of an edge optimization framework in an embodiment;

[0063] Figure 5 A schematic diagram of a building instance segmentation and regularization system based on edge optimization in an embodiment;

[0064] Figure 6 Schematic diagram of a feature fusion process based on deformable convolution in an embodiment;

[0065] Figure 7 Schematic diagram of an electronic device in an embodiment. DETAILED DESCRIPTION

[0066] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0067] Example 1

[0068] In view of the problems existing in the above-mentioned prior art, this embodiment provides a method for segmenting and regularizing building instances based on edge buffers. The framework diagram of the method is shown in FIG. Figure 2 As shown, the method provides a Poly-BRBLE framework that combines a single building location and extraction model based on edge buffer (BRBLE) and a contour regularization post-processing method to achieve a complete process from high spatial resolution aerial images to building vector basemaps.

[0069] See also Figure 1 , the method comprises the following steps:

[0070] Step S1, constructing a high-resolution building image dataset of different scenes.

[0071] Crop the collected high-resolution remote sensing images and convert the corresponding annotations into JSON format to achieve a one-to-one correspondence between image names, building locations, and contours. Regularize the training images to control the distribution of data and ensure the stability of training.

[0072] Step 2: Input the images and labels in the training set into the edge buffer-based single building location and extraction model BRBLE for training to obtain the weight information required for model reasoning. The following will explain the structure of the BRBLE model from two aspects: feature extraction and propagation and building edge optimization, including steps S201-S202:

[0073] Step S201, feature extraction and propagation process.

[0074] The backbone feature extraction network (such as Resnet101, Resnet50, etc.) is used for feature extraction, and the improved feature fusion and propagation structure is used to complete the multi-scale feature fusion and cross-layer feature fusion and transmission of the region of interest. The specific framework diagram is as follows Figure 3As shown in Figure 2. Deformable convolution is introduced in the multi-scale feature fusion process to reduce the inconsistency between multi-scale feature layers. The calculation process is as follows:

[0075] F i =f a (D i +S i ,f Δ (D i +S i ))

[0076] Among them, F i represents the i-th layer feature, D i represents the deep features after upsampling, S i represents shallow features, f Δ Used to calculate the characteristic error, f a Used for feature alignment.

[0077] The traditional ROI feature fusion and propagation process obtains the number of the corresponding feature layer according to the proposed box scale. Since this method ignores other feature layers, resulting in the loss of feature details, this embodiment designs a feature fusion and propagation process that combines spatial and channel attention mechanisms:

[0078]

[0079] in, and Respectively represent the features of the region of interest at the i-th and i-1-th layers, f sigmoid Represents the Sigmoid normalization process, f de represents the feature dimensionality reduction process, f se Represents the calculation process of the channel attention mechanism.

[0080] Step S202: Building edge optimization process.

[0081] In order to improve the blurred building edge problem after instance segmentation, a building edge optimization process is proposed. The process framework is as follows: Figure 4 As shown in the figure. Through the newly added semantic segmentation head, more detailed global semantic segmentation information is obtained, which provides more semantic details for the mask upsampling process with low-detail information. Therefore, an instance feature and semantic feature fusion module SAIU including the upsampling process is provided.

[0082] Taking the instance feature of scale (B, C, H, W) as an example, the corresponding semantic feature scale after cropping is (B, C′, H, W). In order to limit the number of model parameters, the instance feature fused by SAIU has higher spatial resolution and lower number of channels, so C and C′ are not equal during the upsampling process (excluding the initial state), and 1*1 convolution needs to be added to the semantic feature layer to align the number of channels.

[0083] The input of the SAIU module consists of four parts: instance segmentation features Instance segmentation prediction Semantic segmentation features Semantic segmentation prediction The instance segmentation features and predictions are combined into Semantic segmentation features and predictions are combined into Reduce and The number of channels is denoted as and And introduce deformable convolution adjustment spatial information, reducing spatial misalignment and The fusion result is recorded as The upsampled features are Go to the next SAIU module.

[0084]

[0085] Among them, FAM() represents the feature fusion process based on deformable convolution. The specific process is as follows:

[0086] Through the convolution operation f o Get and The spatial misalignment information Δ i-1 .

[0087]

[0088] for Each pixel passes through Make a correction, where ω n represents the weight of the convolution kernel, Representative n Finally, the adjusted features are combined with Add.

[0089] By observing the probability distribution of the building prediction results, we found that: although there is a clear linear boundary between buildings and non-buildings in the image, the prediction result map shows a surface transition zone with a gradually decreasing probability from buildings to non-buildings, which is called the "edge buffer". Compared with linear edges, edge buffers can incorporate more detailed information and environmental information, and focus the model on the difficult-to-distinguish building boundaries. Obviously, the edge buffer can also be regarded as a collection of points (p) suspected of being buildings, and the probability distribution of these points is around 0.5. Combining the multi-layer perceptron (MLP) network and the shallow feature information corresponding to the p point, the probability value of the suspected point can be corrected twice, further improving the model's ability to handle the edges of buildings.

[0090] Optimization of edge points and edge buffer. The operation process is as follows: extract the building edge based on the Laplace operator, set the edge width to w, and obtain the edge buffer. Then replace the building edge buffer of round i-1 with the predicted result of the building edge buffer of round i, and the result is recorded as calculate The K points with the highest uncertainty are input into the multi-layer perceptron (MLP) to make a secondary correction to the prediction results. The corrected results are used as Input SAIU and enter the next round of upsampling.

[0091] Step S3, after obtaining the mask of the building, based on the "main direction-based building regularization algorithm" proposed by Wei et al., considering that the cutting of the building outline by the detection frame will affect the judgment of the "main direction of the outline", an improved outline regularization algorithm is proposed. The following is a specific building regularization process, which specifically includes S301-S304.

[0092] Step S301, using OpenCV to extract the building outline, and using the Douglas-Peucker Algorithm to simplify the edge points, the simplified edge contour curve is recorded as: S1.

[0093] Step S302, calculating the lengths of all line segments in the edge contour curve S1 that do not overlap with the rectangular frame, and recording the direction of the longest side as the "main direction".

[0094] Step S303, calculate the angle between all remaining side lengths and the "main direction", and consider the line segments with angles within 45 degrees as "parallel to the main direction", and the line segments with angles between 45 and 90 degrees as "perpendicular to the main direction". Keep the main direction line segments unchanged, and rotate the above line segments to be parallel or perpendicular to the main direction. The adjusted edge contour curve is recorded as: S2.

[0095] Step S304: The adjusted contour curve S2 is difficult to close, so the line segments are extended or increased according to the distance between adjacent line segments. The adjusted edge contour curve is recorded as: S3.

[0096] Step S4: stitching large scene building detection results.

[0097] Considering that directly predicting the vector outline of buildings in large scene images has extremely high requirements for computer computing power, storage space and other hardware equipment, this embodiment adopts the processing method of "cropping first, then splicing". In the process of image preprocessing, a certain overlapping interval is set to crop the building image of the large scene into an image block of N×N size, and then the building positioning and extraction on the image block are realized through step S2, and finally the spliced ​​and screened building mask is converted into a vector outline through step S3. The splicing process of the building image block is described below:

[0098] Process the building instance segmentation results of the image blocks (i.e., the instance segmentation results of the BRBLE model), average the building probabilities in the overlapping areas, and merge the same buildings distributed in different image blocks. The merging process is: create a new matrix Z with the same scale as the large scene image o , store the spliced ​​building instance segmentation data. Loop each building object and determine the object in Z o If there is an overlapping area on the Z, merge the building objects with IoU greater than ω and assign new numbers to the building objects with IoU less than ω. o When multiple objects overlap, all the objects involved are merged into one object, and the minimum value of their numbers is calculated and used as the number of the merged building object.

[0099] Cycle Z o For each building on the scene, execute step S3 to obtain the regularized outline of the building in the large scene.

[0100] In summary, this method has the following characteristics:

[0101] (1) Aiming at the current demand for high-precision automated vector mapping of buildings, a building instance segmentation and regularization method based on edge buffer is designed. This method can effectively improve the quality of building masks and realize the production of building maps in large scenes.

[0102] (2) A multi-scale feature fusion and propagation module is designed to effectively preserve and propagate feature information of different scales, reducing the loss of feature details caused by traditional feature fusion and propagation models.

[0103] (3) To address the problem of poor edge details in current building masks, a secondary optimization process based on edge buffer is designed, which fully considers the structural characteristics of building edges in terms of points, lines, and surfaces, and effectively retains the edge details of buildings.

[0104] (4) An improvement is proposed for the current mainstream "building regularization algorithm based on main direction" to correct the erroneous main direction caused by the building positioning frame, effectively improving the visualization effect of the building vector results.

[0105] Example 2

[0106] Based on Example 1, see Figure 5 This embodiment provides a building instance segmentation and regularization system based on edge optimization, which is used to implement the building instance segmentation and regularization method of embodiment 1, including:

[0107] (1) Dataset construction module, used to obtain building image datasets in multiple scenes;

[0108] (2) A model building module, which is used to build and train a building location and extraction model based on edge buffers based on building image datasets;

[0109] (3) A contour regularization processing module is used to use the trained building location and extraction model to predict the mask information of the building in the target building image, and to convert the mask information of the building into regularized vector contour information by determining the main direction of the contour.

[0110] Preferably, the model building module includes:

[0111] The feature extraction network is used to extract multi-scale features based on the input building image data, and obtain global semantic segmentation features by multi-scale feature fusion through deformable convolution. The spatial and channel attention mechanisms are used to fuse and propagate the features of the region of interest to obtain instance features. Based on the global semantic segmentation features and instance features, semantic segmentation predictions and instance segmentation predictions are obtained respectively.

[0112] The building edge optimization unit is used to obtain fused features through merging and deformable convolution based on global semantic segmentation features, instance features, semantic segmentation predictions and instance segmentation predictions. Based on the fused features, the instance features for the next building edge optimization are obtained through upsampling. Based on the predictions corresponding to the fused features, the building edge is extracted through the Laplace operator, the uncertainty of the midpoint of the building edge buffer is calculated, and a preset number of points with high uncertainty are used as input. The prediction results are corrected by using a multi-layer perceptron, and the corrected predictions are used as the instance segmentation predictions for the next building edge optimization. Based on the results of multiple building edge optimizations, the mask information of the building is obtained.

[0113] This system has the following features:

[0114] (1) The building instance segmentation model based on edge buffer can effectively improve the edge features of building extraction results and achieve good visual effects under different vectorization post-processing methods.

[0115] (2) The feature fusion and propagation process and the edge optimization process can be applied to different instance segmentation models and have strong transferability.

[0116] (3) The building instance segmentation and regularization method based on edge buffer can realize the production of large-scale building vector maps and has a wide range of application scenarios.

[0117] Example 3

[0118] This embodiment provides an electronic device, including: one or more processors and a memory, wherein the memory stores one or more programs, and the one or more programs include instructions for executing the building instance segmentation and regularization method as described in Example 1.

[0119] like Figure 7 As mentioned above, at the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and may also include other hardware required for the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 1 Of course, in addition to the software implementation, the present invention does not exclude other implementations, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0120] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0121] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0122] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for segmenting and regularizing building instances based on edge optimization, characterized in that: The steps include: Obtain multi-scene building image datasets; Based on the building image dataset, training a building positioning and extraction model based on edge buffer; The trained building location and extraction model is used to predict the mask information of the building in the target building image. The mask information of the building is converted into regularized vector contour information by judging the main direction of the contour.

2. The method for segmenting and regularizing building instances based on edge optimization according to claim 1, characterized in that: The building location and extraction model is configured as follows: Feature extraction and propagation: According to the input building image data, multi-scale features are extracted, and the global semantic segmentation features are obtained by multi-scale feature fusion through deformable convolution. The feature fusion and propagation of the region of interest are performed by combining the spatial and channel attention mechanisms to obtain instance features. Based on the global semantic segmentation features and the instance features, semantic segmentation prediction and instance segmentation prediction are obtained respectively; Building edge optimization: Based on the global semantic features, instance features, semantic segmentation predictions and instance segmentation predictions, a fusion feature is obtained based on deformable convolution; Based on the fused features, instance features for next building edge optimization are obtained by upsampling; Based on the prediction corresponding to the fusion feature, the building edge is extracted by the Laplace operator, the uncertainty of the midpoint of the building edge buffer is calculated, a preset number of points with high uncertainty are used as input, and the prediction result is corrected by using a multi-layer perceptron, and the corrected prediction is used as the instance segmentation prediction for the next building edge optimization; Based on the results of multiple building edge optimizations, the mask information of the building is obtained.

3. The method for segmenting and regularizing building instances based on edge optimization according to claim 2, characterized in that: The multi-scale feature fusion by deformable convolution to obtain the global semantic segmentation feature is based on the following formula: F i =f a (D i +S i ,f Δ (D i +S i )) Among them, F i represents the i-th layer feature, D i represents the deep features after upsampling, S i represents shallow features, f Δ Used to calculate the characteristic error, f a Used for feature alignment.

4. The method for segmenting and regularizing building instances based on edge optimization according to claim 2, characterized in that: The joint space and channel attention mechanism performs feature fusion and propagation of the region of interest, and the instance feature is obtained based on the following formula: in, and Respectively represent the features of the region of interest at the i-th and i-1-th layers, f sigmoid represents the Sigmoid normalization process, f de represents the feature dimensionality reduction process, f se Represents the calculation process of the channel attention mechanism.

5. The method for segmenting and regularizing building instances based on edge optimization according to claim 2, characterized in that: The fusion feature obtained by merging and deformable convolution is based on the following formula: in, They are instance segmentation features, instance segmentation predictions, semantic segmentation features, and semantic segmentation predictions. is the fusion feature of the i-1th building edge optimization, They are the merged instance segmentation features and predictions, the merged semantic segmentation features and predictions, They are Features after reducing the number of channels, concat() represents channel concatenation, FAM() represents feature fusion based on deformable convolution, Upsample() represents upsampling, and Predictor() represents the prediction operator.

6. The method for segmenting and regularizing building instances based on edge optimization according to claim 1, characterized in that: The process of converting the mask information of the building into vector outline information includes: Based on the mask information, extract the building outline and simplify the edge points to obtain a simplified edge outline curve S1; Calculate the lengths of all line segments in the edge contour curve S1 that do not overlap with the rectangular frame, and take the direction of the longest line segment as the main direction; Calculate the angle between each line segment and the main direction, and rotate each line segment to be parallel to or perpendicular to the main direction according to the angle, so as to obtain an adjusted edge contour curve S2; Based on the adjusted edge contour curve S2, the line segment length is adjusted according to the distance between adjacent line segments to close the contour curve and obtain vector contour information.

7. The method for segmenting and regularizing building instances based on edge optimization according to claim 1, characterized in that: The process of obtaining a multi-scene building image dataset includes: Obtain building image data of multiple scenes and corresponding annotation information; Cropping the building image data, converting the annotation information into a preset format, and achieving correspondence between the image name, building location and outline; Regularize for training effects; Construct a multi-scene building image dataset.

8. The method for segmenting and regularizing building instances based on edge optimization according to claim 1, characterized in that: The mask information of the building in the target building image predicted by the trained building positioning and extraction model includes: Acquire an image of a scene to be detected, and obtain a plurality of image blocks in a partially overlapping manner; Using the image block as input to the trained building location and extraction model to predict building classification information and mask information; Based on the corresponding building classification information of the image blocks, the building probabilities in the overlapping areas are averaged and the same buildings distributed in different image blocks are merged; For each merged building, the mask information of the building is converted into a regularized vector outline. The process of averaging the building probabilities in the overlapping areas and merging the same buildings distributed in different image blocks includes: Constructing a matrix matching the size of the scene image to be detected, determining, for each building object in the image block, whether there is an overlapping area at the corresponding position of the matrix, merging building objects whose intersection-and-union ratio is greater than a preset value, and assigning new numbers to building objects whose intersection-and-union ratio is less than the preset value; When the building object of the image block overlaps with multiple objects on the matrix, the multiple objects and the building object of the image block are merged into one object, and the minimum value of the serial number is used as the serial number of the merged object.

9. A building instance segmentation and regularization system based on edge optimization, characterized in that: include: Dataset construction module, used to obtain building image datasets of multiple scenes; A model building module, used for building and training a building positioning and extraction model based on an edge buffer zone based on the building image dataset; The contour regularization processing module is used to use the trained building positioning and extraction model to predict the mask information of the building in the target building image, and convert the mask information of the building into regularized vector contour information by judging the main direction of the contour.

10. The building instance segmentation and regularization system based on edge optimization according to claim 9, characterized in that: The model building module includes: A feature extraction network is used to extract multi-scale features according to the input building image data, obtain global semantic segmentation features by multi-scale feature fusion through deformable convolution, fuse and propagate the features of the region of interest by combining the spatial and channel attention mechanisms, and obtain instance features, and obtain semantic segmentation prediction and instance segmentation prediction based on the global semantic segmentation features and the instance features, respectively; The building edge optimization unit is used to obtain fused features through merging and deformable convolution based on the global semantic segmentation features, the instance features, the semantic segmentation predictions and the instance segmentation predictions; based on the fused features, obtain the instance features of the next building edge optimization through upsampling; based on the predictions corresponding to the fused features, extract the building edge through the Laplace operator, calculate the uncertainty of the midpoint of the building edge buffer, use a preset number of points with high uncertainty as input, use a multi-layer perceptron to correct the prediction results, use the corrected predictions as the instance segmentation predictions for the next building edge optimization, and obtain the mask information of the building based on the results of multiple building edge optimizations.

Images