A method for building segmentation in remote sensing images based on geometric contour vertex prediction

Abstract

The invention discloses a remote sensing image building segmentation method based on geometric contour vertex prediction. The method first uses a two-stage detection network structure, extracts feature information of buildings from remote sensing images through the network, uses ROI‑Align operation to operate the building detection frame to obtain a feature map of the building, and then uses a long short-term memory network based on an attention mechanism on the feature map to model and predict the building contour. In the contour prediction process, edge detection and key point detection are first performed, and the vertex with the highest key point score is used as the starting vertex, which is input into the long short-term memory network based on the attention mechanism to start prediction, and the prediction ends when an end mark is encountered or a maximum time series length is reached. Finally, the predicted vertex results are mapped to the original image, and are connected in sequence to obtain the contour of the building.

Description

Technical Field

[0001] The invention belongs to the technical field of image processing, and in particular to a method for modeling building outlines based on sequence relations. Background Art

[0002] The task of building segmentation is to process remote sensing images, classify the images pixel by pixel, segment each building and obtain the building's outline information. As a hot issue in the field of computer vision, building segmentation is an important research direction in remote sensing image interpretation. The extracted building information is an important data support for the acquisition and update of basic geographic data; and the building segmentation model has a wide range of applications in urban planning, map updating, disaster assessment and other fields.

[0003] Existing building segmentation methods usually adopt a two-stage method of first segmentation and then contour extraction. First, the pixel-level segmentation results of the building are obtained through the image segmentation method based on deep learning, and the border information of each building is obtained. Then, the contour is fitted or modeled to obtain fine building contour information. Although the deep learning-based method can obtain excellent semantic information and edge information, the resolution of remote sensing images is lower than that of natural scenes, and the building contours extracted by the segmentation mask are complex and not smooth enough. They cannot express the geometric characteristics of the building, cannot be directly used for downstream applications, and are not in line with people's direct understanding. The building contour fitting algorithm requires a priori designed manual algorithm, which has a complex process and a long calculation time. The contour fitting algorithm is carried out in stages with the previous segmentation algorithm, cannot be trained end-to-end, and the fitting results are heavily dependent on the previous segmentation results. Therefore, the building contour extracted by the building contour fitting algorithm is not stable enough and has low versatility. The building contour modeling algorithm needs to sample a fixed number of points in advance. Due to the different number of corner points of the building, it is easy to have redundant points, making the boundary line redundant, so that the most human-friendly description of the building cannot be obtained. The above problems lead to the low accuracy of the building contour extracted by the building segmentation model, which needs to be further improved. Summary of the invention

[0004] In order to solve the shortcomings of the above-mentioned prior art, the present invention proposes a remote sensing image building segmentation method based on geometric contour vertex prediction, which can solve the problem of low building segmentation accuracy index caused by fuzzy or uneven building boundaries predicted by the segmentation model in the building segmentation task and redundant building points and edges. The specific technical solution of the present invention is as follows:

[0005] A remote sensing image building segmentation method based on geometric contour vertex prediction, characterized in that it includes the following steps:

[0006] S1: Read the remote sensing image of the area, pre-process and split all the data into 512×512 pixels, and obtain the training set and test set;

[0007] S2: Build a network model, including a building detection module and a contour prediction module;

[0008] S3: Train the network model to obtain a building segmentation model;

[0009] S4: Input the unsegmented remote sensing image into the trained building segmentation model to obtain the result.

[0010] Furthermore, in step S2, the building detection module uses a two-stage detection model, uses a transformer-based feature extractor to extract features, obtains the features extracted by each layer of the extractor to obtain a feature pyramid; uses a region generation network to extract candidate boxes; and uses an RCNN head to classify and regress the candidate boxes to obtain a building detection box.

[0011] Furthermore, the contour prediction module in step S2 includes the following steps:

[0012] S2-1: The multi-layer features of the feature pyramid are fused to obtain a richer feature P, and then the detection box is ROI-Aligned on the feature P to obtain the feature R corresponding to the detection box, and the feature R is mapped to a size of 28×28 to obtain higher resolution features;

[0013] S2-2: Perform edge detection and key point detection on the detection box, take the key point with the largest prediction probability as the starting point, and input it into the contour prediction module to start contour prediction.

[0014] The feature fusion in step S2-1 is specifically as follows:

[0015] The multi-layer features generated by the feature pyramid network are defined as P = {P 2 ,P 3 ,P 4 ,P 5}, with feature P 3 Based on P, the feature information of other layers is adjusted to the same value as P through pooling operation or linear interpolation. 3 The same scale, then add and average the features to get a richer feature information layer P 3 , using the detection box extracted by the building detection module in the feature P 3 Perform ROI-Align operation to obtain the features corresponding to the detection box, map the extracted features to 28×28 size, and obtain high-resolution features R N ×256×28×28 .

[0016] Furthermore, the starting point prediction in step S2-2 is specifically:

[0017] Use two 3×3 convolutions to extract the feature map features obtained by S2-1, use group normalization to assist training, and use two 1×1 convolutions to obtain the building boundary mask E and the building vertex prediction V respectively. After V is activated by sigmoid, the probability of each building point is obtained, and the point with the maximum probability is selected as the starting point S.

[0018] Furthermore, the contour prediction in step S2-2 is specifically as follows:

[0019] The vertex position y of the current time series t Predictive Modeling:

[0020] P(y t |y t-1 ,y t-2 ,y 0 )(1)

[0021] Among them, y 0 is the starting point position, y t-1 is the predicted position at the previous moment, y t-2 Predict the position for the last previous moment.

[0022] The position information of the vertices is concatenated on the feature map obtained in S2-1, including two layers of feature maps with x-coordinate information and y-coordinate information, whose value range is (-1, 1). The value is uniformly taken according to the scale of the feature map, and then two 3×3 convolutions are performed to extract the feature F;

[0023] Use the sigmoid function to activate E and V, obtain reference prediction information, concatenate E, V, F and the three prediction results, use two 64-dimensional long short-term memory networks based on the attention mechanism to extract spatiotemporal features, and use two fully connected networks to predict categories and biases respectively. When the network senses that it needs to predict the first position again, it outputs the end mark to complete the prediction.

[0024] During the training process, each predicted result is replaced with the real result. The predicted result is used during testing and mapped into a binary image. For the first time series, a feature map with all zeros is used instead.

[0025] Furthermore, in step S2, the overall loss of the model is:

[0026] L=L edge +λ vertex *L vertex +L offset +L rnn +L det (2)

[0027] Among them, Ledge represents the building edge prediction loss, L vertex represents the building vertex prediction loss, L offset represents the bias regression loss in temporal vertex prediction, L rnn represents the classification loss in temporal vertex prediction, L det represents the building detection loss, and the building vertex prediction loss weight λ vertex = 10, using overall loss supervision to train the building segmentation model.

[0028] Furthermore, the building vertex prediction loss L vertex Specifically:

[0029] In the building vertex prediction process, the attenuation coefficient is modeled using 2D Gaussian distribution, and the formula is as shown in (3), where offsetx and offsety represent the offset of the sample relative to the ground truth.

[0030]

[0031] Sigmoid activation is used to indicate the probability that a feature point is a building vertex, and the vertex prediction loss L is calculated using the attenuation coefficient and Focalloss loss. vertex , the formula is shown in (4),

[0032]

[0033] Among them, γ is the focusing parameter, β is the weight factor, p represents the probability that the predicted sample belongs to 1, the value of p is 0-1, and θ is the attenuation coefficient.

[0034] Furthermore, the temporal vertex prediction uses a long short-term memory network based on the attention mechanism to predict the vertex positions in sequence, and the vertices are quantized to obtain the coordinates (x, y). Assuming that the feature map scale is w×h, the category is obtained according to the following formula (5):

[0035] c=x+y×w (5)

[0036] Use multi-class cross entropy loss for supervision and add a weight parameter. Except for the number of building vertices and an end flag, the weight is set to 1, and the rest are not involved in training and are set to 0.

[0037] A bias regression loss is added to the temporal vertex prediction loss, and the loss target is shown in formula (6):

[0038]

[0039] o x and yis the learned offset parameter, x and y are the floating point positions of the sample points mapped to the feature map, the target interval is (0,1), and supervised learning is performed using Smooth L1 loss.

[0040] Beneficial effects of the present invention:

[0041] This method adds a building contour prediction module to the existing building segmentation network, which improves the accuracy of building segmentation. The building boundaries predicted by this segmentation model in the building segmentation task are clear and smooth, and no redundant building points and edges are predicted. The building segmentation accuracy (AP) index is high. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The present invention may be better understood by reference to the accompanying drawings, wherein the components in the figures are not to be considered to scale, emphasis instead being placed upon illustrating the principles of the invention.

[0043] Figure 1 Framework of building segmentation model based on geometric contour vertex prediction

[0044] Figure 2 Feature pyramid structure

[0045] Figure 3 Vertex Predictive Modeling - Gaussian Distribution

[0046] Figure 4 Example of building footprint prediction process

[0047] Figure 5 Building outline prediction results DETAILED DESCRIPTION

[0048] In order to facilitate those skilled in the art to understand and implement the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments:

[0049] A building segmentation method based on geometric contour vertex prediction, the process is as follows:

[0050] S1: Read the remote sensing image of the area, pre-process and split all the data into 512×512 pixels, and obtain the training set and test set;

[0051] S2: Build a network model, including a building detection module and a contour prediction module;

[0052] S3: Train the network model to obtain a building segmentation model;

[0053] S4: Input the unsegmented remote sensing image into the trained building segmentation model to obtain the result.

[0054] In terms of data sources, the image data is first divided into training sets and test sets to prevent the training set and test set from coming from the same image, which would cause misleading performance. Since the original remote sensing image is large in scale and cannot be input into the network at one time, all data is preprocessed and segmented into 512×512 pixels. In order to ensure the integrity of building instances as much as possible, there is an overlap of 64 pixels during the segmentation process, and buildings with a ratio greater than 0.5 are retained.

[0055] The building segmentation model is trained based on the Pytorch framework. In the building segmentation network, except for the backbone network, the other networks are randomly initialized using a Gaussian distribution with a mean of 0 and a variance of 0.01. The training process uses the SGD optimizer, with the initial learning rate set to 0.005, the momentum set to 0.9, and the weight decay parameter set to 0.0005. Random horizontal flipping is used for data enhancement, and the training batch size is set to 2. The model is trained for a total of 24 iterations, with the learning rate decayed by 10 times in the 16th and 22nd iterations respectively. The image size is set to a minimum input scale of 800 and a maximum of 512 for training, and the same configuration is used for testing.

[0056] The specific building segmentation framework based on geometric contour vertex prediction can be found in Figure 1 , where the building detection module and the contour prediction module are specifically:

[0057] Building detection module,The building detection stage uses a two-stage detection model. First, a transformer-based feature extractor is used to extract features, then a region generation network is used to extract candidate boxes, and finally the RCNN head classifies and regresses the candidate boxes to finally obtain the building detection box.

[0058] The building outline prediction module can be divided into three parts:

[0059] (1) Feature fusion module. The feature pyramid network generates multiple layers of features, defined as P = {P 2 ,P 3 ,P 4 ,P 5}. The underlying feature P 2 Contains more position information and is more sensitive to local gradient changes; while high-level features P 5 It contains richer semantic information, has a larger receptive field, and can model the entire building. If only single-layer feature information is used, it may cause information loss and have a certain impact on the prediction results. 3 Based on P, it is 1 / 8 of the original image, which can take into account both semantic information and position information, and adjust the feature information of other layers to the same level as P through pooling operations or linear interpolation. 3The same scale, and the features are added and averaged to obtain a richer feature information layer P 3 Then use the detection box extracted in the previous stage to add feature P 3 Perform ROI-Align operation. Unlike the detection network, point prediction requires a higher resolution, so the obtained features are mapped to a size of 28×28 to obtain feature R N×256×28×28 .

[0060] (2) Starting point prediction. The feature map obtained in the previous stage is used to extract features using two 3×3 convolutions, and then two 1×1 convolutions are used to obtain the building boundary mask E and the building vertex prediction V. After V is activated by sigmoid, the probability of each building point can be obtained, and then the point with the maximum probability is selected as the starting point S.

[0061] (3) Contour prediction. First, the position y is predicted at the previous moment t-1 and the predicted position y at the last moment t-2 These two predicted positions can determine the direction of the current time series prediction position. Secondly, when the network senses that it needs to predict the first position again, it can output an end flag to complete the prediction. Therefore, the vertex position y of the current time series t The prediction modeling is shown in formula (7), and the process example is as follows Figure 4 shown.

[0062] P(y t |y t-1 ,y t-2 ,y 0 ) (7) Among them, y 0 is the starting point position, y t-1 is the predicted position at the previous moment, y t-2 Predict the position for the last previous moment.

[0063] The position information is spliced ​​on the basis of the feature map obtained by ROI-Align. The two layers of feature maps represent the x-coordinate information and the y-coordinate information respectively, with a value range of (-1, 1). The value is uniformly taken according to the scale of the feature map, and two 3×3 convolutions are performed to extract the feature F. Each feature F uses group normalization (Group Norm) to help training. During the training process, each prediction result is replaced with the real result. The prediction result is used during testing, and the prediction result is mapped to a binary image. The sigmoid function is used to activate E and V to obtain the reference prediction information, splice E, V, F and the three prediction results, and use two long short-term memory networks based on the attention mechanism with a dimension of 64 to extract spatiotemporal features. For the first time series, since there is no y t-2, so a feature map with all zeros is used instead, and finally two fully connected networks are used to predict the category and bias respectively. It is worth noting that in the contour prediction stage, since all points are trained equally in the previous stage, any point can be used as the starting point, so the number of true value sample points is first randomly rotated to cope with this situation.

[0064] The building detection module uses a two-stage detection model. The loss of the building detection module is the building detection loss, which includes the region generation network loss and the detection head loss. The contour prediction module loss mainly includes the building edge prediction loss, the building vertex prediction loss and the temporal vertex prediction loss. The temporal vertex prediction loss includes the bias regression loss and the classification loss. Using the detection loss to supervise feature extraction can obtain more abstract and representative building features.

[0065] In the process of building edge prediction, the building edge is mapped to the feature map and considered as a positive sample. The rest is regarded as the background. The sigmoid activation function is used to judge the probability that the current feature point is the edge of the building. Therefore, the building edge prediction loss uses the binary cross entropy loss.

[0066] In the process of building vertex prediction, the positions where the real vertices are mapped to the feature map scale and quantized are used as positive samples, and the rest are used as negative samples. Positive samples are still very small compared to negative samples. In order to ensure training balance, Focalloss loss is used, as shown in formula (8).

[0067]

[0068] Among them, γ is the focus parameter, α is the weight factor, p represents the probability of the predicted sample belonging to 1, and the value of p is 0-1, label represents the sample label, and the value of label is {+1,-1}. α=0.25, γ=2, and p is the probability after sigmoid activation.

[0069] Through the above loss, we can prevent a large number of simple sample losses from occupying a dominant position, so as to focus on difficult samples. In addition, because it is acceptable and reasonable to predict a higher score near the true vertex, if all background samples are penalized, there may be a pulling phenomenon near the positive sample during the training process, making it difficult to predict the positive sample. Therefore, the background samples are not punished equally, and an attenuation coefficient is set. The closer to the true vertex position, the larger the attenuation coefficient, which reduces the images with high prediction scores near the true vertex. Specifically, the attenuation coefficient is modeled using a 2D Gaussian distribution, and the formula is as shown in (3), where offsetx and offsety represent the offset of the sample relative to the ground truth.

[0070]

[0071] Simulate a 10×10 feature map and set the vertex at position (5,5) for modeling and visualization. The result is as follows Figure 3 As shown in the figure, the coefficient of position (5,5) is the highest, and the coefficient of other positions gradually decays until it reaches 0. Sigmoid activation is also used to indicate the probability that a feature point is a building vertex. The vertex prediction loss L is calculated using the decay coefficient and Focal loss. vertex , the formula is shown in (4),

[0072]

[0073] Among them, γ is the focusing parameter, β is the weight factor, p represents the probability that the predicted sample belongs to 1, the value range of p is 0-1, and θ is the attenuation coefficient. The closer to the positive sample, the higher θ is. Therefore, even if the predicted probability is the largest, a lower loss can still be obtained, thereby preventing the pulling phenomenon near the positive sample.

[0074] Use multi-class cross entropy loss for supervision, as shown in formula (8), and train on the number of building vertices and an end sign.

[0075]

[0076] Where N represents the number of samples, C represents the number of categories, The sign function takes 1 if the label of sample n is c, otherwise it takes 0. It represents the probability that sample n belongs to category c.

[0077] In temporal vertex prediction, the long short-term memory network based on the attention mechanism is used to predict the vertex positions in sequence. The vertices need to be quantized to obtain the predicted categories. Assuming that the feature map scale is w×h, the vertex is quantized to obtain the coordinates (x, y), and the category is obtained according to the following formula (5):

[0078] c=x+y×w (5)

[0079] Then we use multi-class cross entropy loss for supervision. The training process requires batch processing, and the number of building vertices is different, so we add a weight parameter. Except for the number of building vertices and an end flag, the weight is set to 1, and the rest are not involved in the training and are set to 0.

[0080] In addition to the above classification loss, the vertex prediction process needs to be quantized to a specific position, but the sample position is generally a floating point number, which will cause a certain loss in positioning accuracy. Therefore, a bias regression loss is added to the temporal vertex prediction loss to obtain a more accurate positioning. The loss target is shown in formula (6):

[0081]

[0082] o x and y is the learned offset parameter, x and y are the floating point locations of the sample points mapped to the feature map. The target interval is (0,1), and supervised learning is performed using Smooth L1 loss.

[0083] In summary, the overall loss of the model is as shown in the following formula (2):

[0084] L=L edge +λ vertex *L vertex +L offset +L rnn +L det (2)

[0085] Among them, L edge represents the building edge prediction loss, L vertex represents the building vertex prediction loss, L offset represents the bias regression loss in temporal vertex prediction, L rnn represents the classification loss in temporal vertex prediction, L det represents the building detection loss, and the building vertex prediction loss weight λ vertex Set to 10 and the rest to 1. The model can be trained end-to-end under the constraint of the overall loss function L.

[0086] The building segmentation model converts the contour results into mask results, and uses the ap, ap@50, ap@75, ap@small, ap@medium, and ap@large evaluation indicators proposed by MS COCO as references. Among them, ap@50 reflects that the building is basically segmented correctly; ap@75 reflects the effect under a higher positioning standard; ap@small is the accuracy of small objects; ap@medium is the accuracy of medium objects; ap@large is the accuracy of large objects; ap is a weighted result from ap@50 to ap@95, reflecting the overall performance indicator.

[0087] Several important parameters need to be noted in the process of building outline prediction. First, since the features after ROI-Align will be used to extract temporal features and predict position information (category) later, the feature scale after ROI-Align will affect the accuracy of contour prediction. Secondly, since the long short-term memory network based on the attention mechanism is used for vertex prediction, the influence of the temporal length of the network should be comprehensively considered. Finally, if the prediction score near a point is high, the points near it should get higher scores, so the position-related information of the pixels should be fully utilized.

[0088] In order to verify the influence of pooling scale, time series length, position information, bias regression branch, and average kernel operation parameters on model performance, relevant experiments were carried out.

[0089] (1) Pooling scale

[0090] The pooling scale refers to the feature scale after ROI-Align, which will be used to extract time series features and predict position information (category) later. When the pooling scale is smaller, the accuracy loss after quantization is greater, and the range that can be represented will be smaller. When the inverse transform is mapped to the original image, it will bring higher accuracy. As shown in Table 1 below, the experiment uses five different scales (7, 14, 16, 28, 32). When the pooling scale is 7, ap@75 is very low, indicating that the positioning effect after quantization is difficult to meet the requirements. When the pooling scale gradually increases, it brings about a 20-point improvement in ap@75. As the scale becomes larger and larger, the performance becomes saturated or even decreases, because a larger pooling scale brings a larger optimization space, making optimization difficult. Therefore, the model parameter uses 28 as the standard pooling scale.

[0091] Table 1 Comparison of different pooling scale effects

[0092]

[0093]

[0094] (2) Sequence length

[0095] The longer the time series is, the more contour points of complex buildings can be retained, but it will increase the training time and over-model complex buildings, resulting in excessive loss, and the optimization falls into the local optimum, affecting the detection accuracy of buildings. As shown in Table 2 below, when the time series length is 5, the performance is quite different from that of other time series lengths; the other time series lengths have little effect on the accuracy, indicating that the number of building vertices can be well covered. Therefore, the model parameter uses 20 as the standard time series length.

[0096] Table 2 Comparison of effects at different time scales

[0097]

[0098] (3) Location information

[0099] In the process of contour prediction, there is a certain correlation between the positions of the predicted points. This method establishes two explicit position information feature maps, which represent the relative positions of the x-coordinate and the y-coordinate respectively. The value range is (-1, 1), and the value is uniformly taken according to the scale of the feature map. By adding two layers of features in the first convolution process, the position information is explicitly incorporated. The experimental results are shown in Table 3 below. Adding position information has a 0.7% gain in AP.

[0100] Table 3. Impact of location information performance

[0101]

[0102] (4) Biased regression branch

[0103] In contour prediction, the labeled data needs to be mapped to the pooled feature map and quantized to obtain the category to be predicted. There is position loss in the quantization process, which affects the positioning accuracy. To solve the above problem, a regression branch is added to the prediction part of the long short-term memory network based on the attention mechanism to predict the loss of the quantization process. The experimental results are shown in Table 4 below. Adding a regression branch does not bring any benefits to ap@50, which is mainly reflected in the ap@75 indicator, that is, there is a greater improvement in positioning accuracy; for object scale, the performance improvement is mainly reflected in medium and large objects. This is because the loss of the quantization process of medium and large objects is more obvious than that of small objects.

[0104] Table 4 Impact of bias regression branch performance

[0105]

[0106] (5) Average kernel operation

[0107] If the prediction score near a point is high, the points near it should get a higher score. In order to make full use of this feature and increase the stability of the inference process, the scores of the points near the prediction result are weighted, and the highest score is selected as the prediction result of the current time series. The weighting method mainly uses Gaussian weighting and sum weighting. This experiment tests the impact of different weightings and weighting ranges on accuracy. Weighting is achieved through convolution, where the Gaussian kernel is a two-dimensional Gaussian kernel function, and the constant means that all weights are 1. The experimental results are shown in Table 5 below. The first row does not use sum weighting, which can be considered as using a Gaussian kernel or a constant kernel with a kernel scale of 1. It can be seen from the table that the constant kernel will seriously reduce the performance because the difference in the center position is not considered at this time, resulting in a serious change in the score distribution. When the kernel scale becomes larger, the distribution changes more drastically, which leads to problems in subsequent predictions. By using the Gaussian kernel, the weight of the center position is high and the weight of the nearby is low, thereby highlighting the center position of the prediction point and obtaining a more stable result. When the Gaussian kernel scale is 3 or 5, the effect is almost the same. To simplify the calculation, the Gaussian kernel scale of 3 is used.

[0108] Table 5 Impact of different core operations

[0109]

[0110] According to the above experimental results, we selected the experimental configuration with pooling scale 28 and time series length 20 for ablation experiment. The results are shown in Table 6 below:

[0111] Table 6 Performance comparison of each module

[0112]

[0113] 5) Comparison with other methods

[0114] Table 7 reports the comparison of the results of the proposed method and Mask RCNN on the public dataset. In order to compare the fitting results of the building outline, the segmentation results of Mask RCNN are post-processed. In Table 7, Mask RCNN (rectangle) means that the polygon result is represented by the minimum enclosing rectangle, and Mask RCNN (polygon) means that the mask result is fitted into a polygon using the fitting algorithm. This method can directly obtain the polygon result through the long short-term memory network based on the attention mechanism without post-processing operations. As can be seen from the table, the accuracy of this method is higher than that of the fitting algorithm, which is about 6.1% higher, which proves the effectiveness of the proposed method.

[0115] Table 7 Comparison of the proposed method with other methods

[0116]

[0117] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A remote sensing image building segmentation method based on geometric contour vertex prediction, characterized in that: The following steps are involved: S1: Read the remote sensing image of the area, pre-process and split all the data into 512×512 pixels, and obtain the training set and test set; S2: Build a network model, including a building detection module and a contour prediction module; S3: Train the network model to obtain a building segmentation model; S4: Input the unsegmented remote sensing image into the trained building segmentation model to obtain the result; The building detection module in step S2 is a two-stage detection model. Use the transformer-based feature extractor to extract features, and obtain the features extracted by each layer of the feature extractor to obtain a feature pyramid; Use region generation network to extract candidate boxes; The RCNN head classifies and regresses the candidate frames to obtain the building detection frame; In step S2, the contour prediction module performs the following steps: S2-1: The multi-layer features of the feature pyramid are fused to obtain a richer feature P, and then the detection box is ROI-Aligned on the feature P to obtain the feature R corresponding to the detection box, and the feature R is mapped to a size of 28×28 to obtain higher resolution features; S2-2: Perform edge detection and key point detection on the detection box, take the key point with the highest prediction probability as the starting point, and input it into the contour prediction module to start contour prediction; The feature fusion in step S2-1 is specifically as follows: The multi-layer features generated by the feature pyramid network are defined as P = {P2, P3, P4, P5}. Based on feature P3, the feature information of other layers is adjusted to the same scale as P3 through pooling operation or linear interpolation, and then the features are added and averaged to obtain a richer feature information layer P3. The detection frame extracted by the building detection module is used to perform ROI-Align operation on the richer feature information layer P3 to obtain the features corresponding to the detection frame. The extracted features are mapped to a size of 28 × 28 to obtain a high-resolution feature R N×256×28×28 ; The starting point in step S2-2 is determined by: Use two 3×3 convolutions to extract the feature map features obtained in step S2-1, use group normalization to help training, and use two 1×1 convolutions to obtain the building boundary mask E and building vertex prediction V respectively. After V is activated by sigmoid, the probability of each building vertex is obtained, and the point with the maximum probability is selected as the starting point S; The contour prediction in step S2-2 is specifically as follows: The vertex position y of the current time series t The prediction modeling formula is as follows: P(and t |and t-1 ,and t-2 ,y0)(1) Among them, y0 is the starting point, y t-1 is the predicted position at the previous moment, y t-2 Predict the position for the last previous moment; The position information of the vertices is concatenated on the feature map obtained in step S2-1, including two layers of feature maps of x-coordinate information and y-coordinate information, whose value range is (-1, 1), and the value is uniformly taken according to the scale of the feature map, and then two 3×3 convolutions are performed to extract the feature F; Use the sigmoid function to activate E and V to obtain reference prediction information; Concatenate E, V, F and the three prediction results, and use two LSTM networks with a dimension of 64 and an attention mechanism to extract spatiotemporal features; Use two fully connected networks to predict the category and bias respectively. When the network senses that it needs to predict the first position again, it outputs the end mark to complete the prediction. During the training process, each predicted result is replaced with the real result. The predicted result is used during the test and mapped into a binary image. For the first time series, a feature map with all zeros is used instead. The overall loss of the model is: L=L edge +λ vertex *L vertex +L offset +L rnn +L det (2) Among them, L edge represents the building edge prediction loss, L vertex represents the building vertex prediction loss, L offset represents the bias regression loss in temporal vertex prediction, L rnn represents the classification loss in temporal vertex prediction, L det represents the building detection loss, and the building vertex prediction loss weight λ vertex =10, using overall loss supervision to train the building segmentation model; In the building vertex prediction process, in order to ensure the balance of positive and negative training samples and avoid the pulling phenomenon near the positive samples during training, the attenuation coefficient is modeled using 2D Gaussian distribution, and the formula is as shown in (3), where offsetx and offsety represent the offset of the sample relative to the ground truth. The sigmoid activation function is used to represent the probability that a feature point is a building vertex, and the vertex prediction loss L is calculated using the attenuation coefficient and Focalloss loss. vertex , the formula is shown in (4), Among them, γ is the focusing parameter, β is the weight factor, p represents the probability that the predicted sample belongs to 1, the value of p is 0-1, and θ is the attenuation coefficient.

2. The remote sensing image building segmentation method based on geometric contour vertex prediction according to claim 1 is characterized in that: Temporal vertex prediction uses a long short-term memory network based on an attention mechanism to predict vertex positions in sequence, and quantizes the vertices to obtain coordinates (x, y). Assuming that the feature map scale is w×h, the category is obtained according to the following formula (5): c=x+y×w (5) Use multi-class cross entropy loss for supervision, the formula is as follows: Where N represents the number of samples, C represents the number of categories, The sign function takes 1 if the label of sample n is c, otherwise it takes 0. It represents the probability that sample n belongs to category c.

3. The remote sensing image building segmentation method based on geometric contour vertex prediction according to claim 2 is characterized in that: A bias regression loss is added to the temporal vertex prediction loss, and the loss target is shown in formula (6): o x′ and y′ is the learned offset parameter, x′ and y′ are the floating-point positions of the sample points mapped to the feature map, the target interval is (0,1), and supervised learning is performed using Smooth L1 loss.

Images