A method for extracting building vector outline data from high-resolution remote sensing images

Abstract

The present invention relates to a method for extracting building vector outline data from high-resolution remote sensing images, aiming to directly extract vector outline data with regular building edges from high-resolution remote sensing images. A multi-task segmentation model is primarily used to predict building masks, building vertex probability distributions, and building vertex angles. Based on these outputs, a rough building vector outline result is obtained and building vector outline feature data is constructed. A vector optimization module based on a graph attention network is further used to optimize the rough vector outline result of the front-end network to obtain the final regular building vector outline data. The present invention is simple and easy to implement, and can directly extract building vector outline data from remote sensing images without the need for human intervention, resulting in a regular building vector outline result.

Description

Technical Field

[0001] The invention belongs to the field of methods for extracting vector outline data of buildings from remote sensing images, and in particular relates to a method for extracting vector outline data of buildings from high-resolution remote sensing images. Background Art

[0002] In recent years, with the rapid development of Earth observation technology, the resolution of remote sensing images has continued to increase, providing a rich source of information. Rapidly extracting and analyzing this information will help us better understand and analyze the urban environment. Buildings are one of the most prominent features in the urban environment and a key indicator of urban change and human activity. Rapid and accurate automatic extraction of large-scale building vector outlines based on high-resolution imagery can provide important data support for applications such as map making, urban planning, land and resources surveys, and earthquake relief, and has broad application prospects.

[0003] Traditional building extraction methods typically consist of feature extraction and label classifiers. These methods design discriminative features for buildings, such as color, texture, shape, and straight lines, and then apply support vector machines (SVMs), random forests, decision trees, or other techniques to the feature information to extract buildings. Because these methods rely heavily on manually designed low-level features, they lack intelligence and only achieve good extraction accuracy on low-resolution remote sensing imagery. With the increasing prevalence of high-resolution remote sensing imagery, these methods are no longer able to adapt to the diversity and complexity of buildings found in high-resolution remote sensing imagery, resulting in problems such as poor algorithm robustness and low accuracy. In recent years, deep learning technology has achieved remarkable results in computer vision tasks. Image segmentation models are widely used for building extraction. Deep neural networks adaptively learn the inherent multi-level feature representation of samples, effectively adapting to the complexity of buildings in high-resolution remote sensing imagery.

[0004] Although deep learning-based building extraction has achieved considerable research results, the wide range of spectral and texture features of buildings in high-resolution remote sensing imagery, as well as the complex and varied shapes of building edge contours, makes existing pixel-level classification models based on convolutional local features suffer from significant classification uncertainty when extracting building edge contours. This makes it difficult to accurately identify building edges, resulting in jagged edges, rounded corners, and difficulty maintaining overall geometric features. Furthermore, since most current mainstream building extraction models extract building raster mask data, while vector data is typically used in practical application analysis, the extracted building mask data must be post-processed to be converted from raster to vector to meet practical application requirements, making end-to-end building vector contour extraction impossible. Furthermore, due to the low accuracy of segmentation models in identifying building edge contours, the vector results after raster to vector conversion are also inaccurate. Finally, most building extraction models require a large number of sample labels for supervised learning. However, vector labeling of buildings in high-resolution remote sensing imagery is time-consuming and labor-intensive, and most datasets lack sample labels or have low label quality, making the models unsuitable for new scenarios. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide a method for extracting vector outline data of buildings from high-resolution remote sensing images.

[0006] The technical solution of the present invention to solve the above technical problems is as follows:

[0007] A method for extracting building vector outline data from high-resolution remote sensing images comprises the following steps:

[0008] Step 1: Obtain the original remote sensing image and the corresponding vector contour label data to form a training data set, and then generate the corresponding mask label, vertex label and angle label according to the vector label data;

[0009] Step 2: Establish a high-resolution remote sensing image building vector contour extraction model based on a graph attention network, obtain training data from the training dataset, and input the training data and the corresponding mask labels, vertex labels, and angle labels into the high-resolution remote sensing image building vector contour extraction model based on a graph attention network;

[0010] The high-resolution remote sensing image building vector contour extraction model based on the graph attention network includes a multi-task segmentation model, a rough building edge vector construction module and a building vector optimization module based on graph attention. The multi-task segmentation model is used to simultaneously obtain the building edge vector angle direction features according to the building mask label, vertex label and angle label building. angle , building mask feature O maskand building vertex distribution characteristics O ver The rough building edge vector outline construction module is used to adopt a method based on edge vector vertex and angle direction sampling, according to the building edge vector angle direction feature O angle , building mask feature O mask and building vertex distribution characteristics O ver Get the rough building vector outline result P boundary The building vector outline optimization module based on graph attention is used to optimize the rough building vector outline result P boundary Optimize and remove redundant coordinate points to make the vector results more regular, and obtain the final optimized building vector outline coordinate set P;

[0011] Step 3: Based on the obtained building vector outline coordinate set P and the training label, the loss value is calculated using the loss function. The building vector outline extraction model based on the graph attention neural network is trained by repeating step 2, and the model parameters are adjusted until the loss function is minimized. The model training is completed, and the final high-resolution remote sensing image building vector outline extraction model based on the graph attention network is obtained.

[0012] Step 4: Extract building vector contours from remote sensing images based on the final high-resolution remote sensing image building vector contour extraction model based on the graph attention network.

[0013] Furthermore, in step 1, generating the angle tag specifically refers to calculating the current position P (x,y) The angle θ∈[-π,π] formed by the vector pointing to the vertex of the building and the x-axis direction is calculated as follows:

[0014] Step 1.1. Obtain an angle label matrix with the same resolution as the mask label. Each value in the angle label matrix represents the angle label corresponding to the vector point with the row and column values ​​as coordinates.

[0015] Step 1.2: Match each vector point in the angle label matrix with a vertex that is closest to it, and inherit the next vertex pointed to by the vertex as the direction of the vector point.

[0016] Furthermore, in step 2, the multi-task segmentation model uses the backbone network Res-U-Net to extract the shared feature map of the image, and then uses three task branches with 256 convolutional layers and ReLU activation functions to extract high-dimensional semantic features in the shared feature map. The three task branches are used to learn the building vertex distribution features, building mask features and building edge vector angle direction features, respectively.

[0017] Furthermore, in step 2, the feature representation of building edge angles and building vertices is enhanced by introducing the ECA module, and finally the building edge vector angle direction feature is obtained by compressing the number of channels through 1x1 convolution. angle , building mask feature O mask and building vertex distribution characteristics O ver .

[0018] Furthermore, in step 2, the method based on edge vector vertex and angle direction sampling includes the following steps:

[0019] Step 2.1a: Use the NMS algorithm to remove vector points that are too close, and then distribute the features of the building vertices. ver Use a sliding window of size S and step size p to sample the building vertex distribution and retain the vertex with the highest probability in the sliding window;

[0020] Step 2.2a: Sort the building vertex distribution after NMS algorithm filtering by probability, and select the first N vertices as the rough building edge vector point P. boundary ;

[0021] Step 2.3a: Use the Hungarian algorithm to obtain the connection order of the coordinate points through the distance cost matrix, and finally obtain the rough edge vector point P containing the connection order. boundary .

[0022] Furthermore, in step 2.3a, the distance cost matrix is ​​composed of the rough building edge vector point P boundary The sine distance between each other is constructed using the Hungarian algorithm. The distance cost matrix is ​​regarded as the cost matrix of task allocation. When the overall cost is minimized, the order of the building vector points can be obtained to be a pairwise correspondence relationship, avoiding repeated connections. Finally, the rough edge vector point P containing the connection order is obtained. boundary .

[0023] Furthermore, the working process of the graph attention-based building vector optimization module includes the following steps:

[0024] Step 2.1b: First, use the rough building vector outline result P boundary The graph structure data of the building edge vector outline is constructed with the corresponding convolution semantic features, where the graph structure feature vector d is fused with the feature vectors corresponding to the edge vectors in the res2 layer and out layer of U-Res-Net and P boundary Coordinate information of

[0025] Step 2.2b: The graph attention network uses multiple layers of graph attention layers to extract graph structure semantic features from graph structure data.

[0026] Step 2.3b, according to the semantic features of the graph structure, the optimization module cls and offset are used to obtain a more regular vector result. The optimization module cls is used to optimize the rough building vector result P according to the semantic features of the graph structure. boundary After removing unnecessary vector points, we can get The optimization module offset is used to optimize the rough building vector result P according to the semantic features of the graph structure. boundary The spatial coordinates are corrected and the eigenvalues ​​are scaled to [0,1] using the sigmoid activation function to obtain the offset vector d offset , then through d offset 2-1 scales the value to [-1,1], and the optimized building edge vector P is given by and d offset Calculated, The hyperparameter γ represents the confidence value of the offset result.

[0027] Furthermore, in step 3, the loss function of the model includes the loss of the building vector optimization module based on graph attention. cls , shape constraint loss l shape and multi-task segmentation model loss; where the multi-task segmentation model consists of the building vertex loss l ver , building edge vector angle direction loss l angle and building segmentation mask loss l mask Composition, total loss l total The calculation formula is as follows:

[0028] l total =λ c ·l cls +λ s ·l shape +λ v ·l ver +λ a ·l angle +λ m ·l mask

[0029] where λ c ,λ s ,λ v ,λ a and λ m is the weight coefficient of the loss function corresponding to each module.

[0030] Furthermore, in step 3, the loss of the multi-task segmentation model is l mask 、l angle and l veris the building segmentation mask O obtained by multi-task prediction mask , building edge vector angle direction O angle and building vertex O ver The corresponding sample labels are obtained through cross entropy loss. The building vector optimization module based on graph attention includes two task branches: optimization module cls and optimization module offset. The optimization module cls selects the result d according to the predicted vector points. cls The binary cross entropy loss is calculated with the sample label. The optimization module offset does not perform separate supervised learning and uses the shape constraint loss function l shape To train, the shape constraint loss function l shape Contains the distance metric loss function l cd and the sinusoidal loss function l sine , that is: l shape =l cd +l sine .

[0031] The beneficial effects of the present invention are as follows: (1) The present invention realizes end-to-end building vector contour extraction from remote sensing images. Compared with the existing building contour extraction based on segmentation models, which can only obtain building contour raster data, the present invention can directly obtain building vector contour data from remote sensing images; (2) The present invention introduces a graph attention network to learn the spatial relationship and contextual features between the building edge vector structure, thereby realizing the overall shape constraint of the building edge vector contour. Compared with the existing building segmentation model, which has great classification uncertainty in the building edge area, the convolution local feature is difficult to take into account the overall shape of the building, resulting in the extracted building contour edge not conforming to the basic geometric form of the building. Based on the multi-task segmentation model, the present invention utilizes the building edge vector sampling scheme based on the classification uncertainty of the segmentation mask in the present invention, constructs the graph structure features of the building edge vector contour from the multi-task segmentation results, considers that the overall shape of the building is affected by the spatial position between the building vector nodes, uses the graph attention network to capture the refined features between the nodes, and uses these features to optimize the spatial position between the building vector points, making up for the blurred segmentation results of the pixel-based convolution local feature in the edge area, and improving the regularity of the building edge vector contour extraction.

[0032] Illustrations

[0033] Figure 1 It is a workflow diagram of the present invention;

[0034] Figure 2 Schematic diagram of training data, where (a) is remote sensing image, (b) is mask label, (c) is vertex label, and (d) is angle label;

[0035] Figure 3Schematic diagram of angle difference for angle labels;

[0036] Figure 4 This is a workflow diagram of the high-resolution remote sensing image building vector extraction model based on graph attention network;

[0037] Figure 5 This is a schematic diagram of the ECA model workflow;

[0038] Figure 6 This is a schematic diagram of the multi-task segmentation model workflow;

[0039] Figure 7 Schematic diagram of the workflow of the building vector optimization module based on graph attention;

[0040] Figure 8 Schematic diagram of the training process of the high-resolution remote sensing image building vector contour extraction model based on graph attention neural network. DETAILED DESCRIPTION

[0041] The principles and features of the present invention are described below with reference to the accompanying drawings. The examples given are only used to explain the present invention and are not used to limit the scope of the present invention.

[0042] like Figure 1 As shown in Figure 2, the specific implementation steps of the high-resolution remote sensing image building vector contour extraction method based on graph attention network are as follows:

[0043] Step 1: Obtain the original remote sensing image and the corresponding training labels to form a training dataset.

[0044] Remote sensing image data and corresponding vector label data that require prior data As the training dataset label, each sequence corresponds to a building instance. Then, the corresponding mask label, vertex label and angle label are generated according to the vector label data, such as Figure 2 As shown, details are as follows.

[0045] Mask label: is a matrix M of the same size as the remote sensing image mask ∈F H×W , where each pixel value is 0 or 1, 0 represents background and 1 represents building. Draw based on vector label data.

[0046] Vertex label: Use the building vertex vector to map to a matrix M of the same size as the remote sensing image ver ∈F H×W As vertex labels, it is used to learn the distribution probability graph of vertices.

[0047] Angle label: Angle label calculation is the current position P (x,y)The angle θ∈[-π,π] formed by the vector formed by it pointing to the vertex of the building and the x-axis direction. Figure 3 As shown, the current position P (x,y) is the origin, and the x-axis is the positive direction. The angle of counterclockwise rotation along the x-axis is [0,π], such as P1 (x1,y1) is the vertex that P points to, then the angle between P and The angle of clockwise rotation along the x-axis is [-π, 0], such as P2 (x2,y2) is the vertex that P points to, then the angle between P and In actual calculation, assuming P1 (x1,y1) is the vertex pointed to by P and is calculated using the following formula.

[0048]

[0049] When generating angle labels, it is necessary to obtain the angle label matrix M with the same resolution as the mask label angle ∈F H×W Each value in the matrix represents the angle label corresponding to the vector point with the row and column values ​​as coordinates. However, to obtain this angle label, we need the vertex information pointed by the vector point. Obviously, this information is only available at the building vertex. Therefore, for each vector point in the matrix, we match the nearest vertex and inherit the next vertex pointed by the vertex as the direction of the vector point. In the algorithm implementation, we first generate the matrix M corresponding to the next vertex of each position. next ∈F H×W×2 , M next Generated using breadth-first search algorithm. Using vertex labels as search starting points Set, all starting points are searched synchronously, and each vector point traversed has the same M as the starting point. next Value, pseudo code is as follows:

[0050]

[0051] Generate M according to the above algorithm next Combining with formula (1), we can get M angle .

[0052] Step 2: Design a high-resolution remote sensing image building vector contour extraction model based on graph attention network, such as Figure 4The graph attention network-based high-resolution remote sensing image building vector extraction model consists of a feature extraction module, a multi-task segmentation module, and a graph attention network-based vector optimization module. The main idea of ​​the graph attention-based building vector contour extraction is to map the local semantic features extracted by the multi-task segmentation model to the building vertex vector points, then use the convolutional semantic features to construct the graph structural features of the building edge vector points. Finally, the graph attention network's ability to learn adjacent features of graph structural data is used to optimize the building vector contour extraction results, thereby improving the regularity of the building vector edges.

[0053] 1) Feature Extraction Module: Res-U-Net is used for feature extraction. Res-U-Net is a U-Net encoder constructed using ResNet modules. ResNet is a large-scale convolutional neural network built using residual modules. The residual module contains two activation functions, two convolutional layers, a batch normalization layer, and skip connections. There are two types of residual modules: Convolutional Block and Identity Block. The Conv Block is used to change the network dimension.

[0054] Res-U-Net uses the Resnet module to construct a U-Net encoder, and then inputs the learned features into the decoder. The encoder network consists of one convolutional layer and four residual layers. After the encoder extracts the features, the upsampling layer inputs the features into the decoder. The residual layers are all composed of residual modules, with a total of four residual layers. The first residual layer consists of 3 residual modules, the second residual layer consists of 4 residual modules, the third consists of 6 residual modules, and the fourth consists of 3 residual modules. Each residual module has two parts: the backbone and the bypass. Figure 5 As shown. The encoder performs feature compression on the data four times, once for each residual layer, and the spatial size of the feature map is halved before being input to the upsampling layer. The upsampling layer uses the UpSampling2D method, which modifies the original tensor by copying and interpolating, which is the inverse operation of the average pooling layer. This can unify the image size before inputting it into the decoder, ensuring that the feature map size of each convolutional layer of the decoder is the same as the size of the symmetric part in the encoder. The internal structure of the encoder is as follows Figure 6 As shown in .

[0055] In the encoder, downsampling is used to halve the size of the feature map and perform feature extraction. The extracted features are transferred from one residual layer to the next. The encoder feature extraction process is then passed to the decoder, where the feature map size is doubled and semantic information is incorporated using skip connections to supplement the low-dimensional semantic features missing from the high-dimensional semantic features. Upsampling is performed at each stage of the decoder.

[0056] The present invention uses Res-U-Net as the backbone network for feature extraction of the multi-task segmentation model. The network extracts high-dimensional semantic features through the encoder, and its output is connected to the output of the previous layer decoder through a jump connection. i Perform feature fusion and finally get d i-1 The present invention uses d0 as the shared feature of the multi-task segmentation model.

[0057] 2) Multi-task segmentation model: The method of the present invention needs to construct rough building edge vector points and corresponding semantic features based on the building vertex distribution, building mask and building edge vector angle direction. If each task is extracted separately, not only the amount of calculation is large, but also the correlation between tasks cannot be used to improve the extraction effect. Therefore, the present invention adopts a multi-task segmentation model to achieve unified extraction. The multi-task segmentation model of the present invention mainly includes three task branches, which are used to learn the building vertex distribution features, building mask features and building edge vector angle direction features, such as Figure 6 The following is a detailed introduction to the extraction process.

[0058] First, the image I∈R 3×W×H Input the multi-task segmentation model and use the backbone network Res-U-Net to extract the shared feature map of the image in s represents the downsampling factor of the backbone network. Next, three convolutional layers with 256 output channels and ReLU activation functions are used to extract high-dimensional semantic features in the shared feature map to obtain the edge direction feature map F of the building. angle , mask feature map F mask and vertex feature map F ver .

[0059] In addition, since there is a strong correlation between the vertex features and edge angle features of the building and the membrane features of the building mask, the channel attention (ECA) module can effectively learn the relationship between different channels in the image and realize soft parameter sharing. Moreover, the final ECA module output does not change the size of the input feature map. Assume that the ECA module input feature map F∈R C ×H×W In the spatial dimension, global average pooling (GAP) is used to obtain a C×1×1 feature map. When performing convolution operations, the convolution kernel size k affects the receptive field. To solve different input feature maps and extract features in different ranges, ECA uses a dynamic convolution kernel to perform 1×1 convolution C1D to learn the importance between different channels. The calculation of k is shown in formula (2), setting γ = 2 and b = 1 to change the ratio between the number of channels C and the convolution kernel.

[0060]

[0061] For the compressed feature map, channel feature learning is performed. Through 1×1 convolution C1D, the importance between different channels is learned. At this time, the output dimension is still C×1×1. Finally, the channel attention is combined. The feature map C×1×1 of the channel attention and the original input feature map C×H×W are multiplied channel by channel, and finally the feature map with channel attention is output. The model diagram of the ECA module is as follows Figure 5 As shown, where C = 256.

[0062] The ECA module is introduced to enhance the feature representation of building edge angles and building vertices. The enhanced building edge angle and vertex features are represented as F′ respectively. ver and F′ angle , the calculation method is shown in formulas (3) and (4). Finally, the 1x1 convolution is used to compress the number of channels to obtain the building edge vector angle direction feature O angle , building mask feature O mask and building vertex distribution characteristics O ver .

[0063] F′ ver =σ(C1D(GAP(F seg +F ver )))·F ver +F ver (3)

[0064] F′ angle =σ(C1D(GAP(F angle +F seg )))·F angle +F angle (4)

[0065] 3) Rough building edge vector construction module: The multi-task segmentation model obtains the building edge vector angle direction features O angle , building mask feature O mask and building vertex distribution characteristics O ver , it is necessary to further obtain rough building edge vector coordinate points based on these feature data to provide basic data for the subsequent building vector optimization module based on graph attention. Most existing methods use building mask edge tracking algorithms to obtain building vector contour data. The premise of this method is that a complete building instance mask result needs to be extracted first. However, when facing dense building groups, building segmentation mask extraction is prone to false detection, resulting in the extracted building edge vector data being unable to distinguish different building instances. To solve this problem, the present invention proposes a method based on edge vector vertex and angle direction sampling. First, based on the building vertex distribution feature O verWe then use the data to obtain N vertices by probability sorting, then calculate the distance cost matrix of the vertices based on the sine distance. Finally, we use the Hungarian algorithm to obtain the connection order of the building vertices and obtain a rough building edge vector. This method will be described in detail below.

[0066] First, select the building vertex distribution O ver However, directly selecting the first N vertices will result in a large number of vertices that are too close, and these repeated extraction results need to be discarded. Therefore, the present invention uses the NMS algorithm to remove vector points that are too close, and then calculates the building vertex distribution feature O ver A sliding window with a size of S and a step size of p is used to sample the building vertex distribution, and the vertex with the highest probability in the sliding window is retained. Then, the building vertex distribution filtered by the NMS algorithm is sorted according to probability, and the first N vertices are selected as the rough building edge vector point P. boundary But P boundary It is just a discrete vector coordinate point, and there is no connection order, so it is impossible to construct graph structure data. The following will obtain the connection order of the coordinate points through the distance cost matrix.

[0067] The distance cost matrix consists of the rough building edge vector points P boundary The sine distance between each other is constructed. First, the sine distance d is obtained, and the calculation method is shown in formula (6). Among them, α and β are weight coefficients, which are used to balance the influence of Euclidean distance and angle on sine distance, and avoid the mismatch caused by the sine distance calculated too small for points with small angles and too far distances; θ represents the minimum angle between the vector direction of point x prediction and the vector (x, y) direction; θ prediction Indicates the angle between the next vertex pointed by point x and the positive coordinate axis; θ x,y Represents the angle of point y relative to point x. For the specific representation of the prediction vector of point x in model training and the calculation method of the minimum angle θ, please refer to the dataset construction section, as shown in formula (5). Then, according to the calculation of P boundary The sinusoidal distance is matched with each other to obtain the distance cost matrix D∈F N , as shown in formula (7).

[0068] θ=(θ prediction -θ x,y +π)modπ (5)

[0069]

[0070] D ij =d(P i boundary ,P j boundary ) (7)

[0071] If the nearest neighbor distance method is used to obtain P based solely on the distance cost matrix boundary If the connection order is too complicated, it is easy for multiple vector points to point to the same vector point, resulting in repeated connections. If the Hungarian algorithm is used, the distance cost matrix is ​​regarded as the cost matrix of task allocation. When the overall cost is minimized, the order of the building vector points can be obtained to be a pairwise correspondence, thus avoiding repeated connections. Finally, a rough edge vector point P containing the connection order is obtained. boundary .

[0072] 4) Building vector optimization module based on graph attention: rough building vector outline result P boundary The present invention uses a building vector optimization module based on graph attention to optimize the rough building vector result P boundary Optimize and get a more regular vector result. The overall model is as follows Figure 7 shown.

[0073] Rough building vector result P boundary The result of the multi-task segmentation model is sampled and calculated using the building vector contour extraction module. It uses pixel-level local convolution features and does not fully exploit the spatial correlation features between building vectors, resulting in poor shape regularity of the extracted building vector results. The building vector optimization module based on graph attention can use the graph attention network to learn the domain relationship of each node and mine the spatial correlation features between vector nodes from the domain relationship. The idea is: first, use the rough building vector result P boundary The graph structure data of the building edge vector is constructed by the corresponding convolutional semantic features, and then the graph attention network is used to extract the graph structure semantic features in the data. Finally, the optimization module is used to obtain a more regular vector result. The specific implementation method is described as follows:

[0074] First, use the rough building vector result P boundary The graph structure data of the building edge vector is constructed with the corresponding convolution semantic features. In order to enrich the feature information, the graph structure feature vector d (Formula 8) is fused with the feature vectors corresponding to the edge vectors in the res2 layer and out layer of U-Res-Net and P boundary coordinate information.

[0075] d=[(d res2 +d out )||P boundary ] (8)

[0076] Then, the graph attention network is used to extract the graph structure semantic features in the data. The graph structure semantic feature extraction part uses a multi-layer graph attention layer to gradually learn the vector space neighborhood features. The input of the graph attention layer is the feature vector The output is a new set of node feature vectors In order to obtain sufficient expressive power, the input features are transformed into higher-level features, that is, a shared linear transformation parameterized by the weight matrix W∈R is applied to each node. F′×F First, the shared attention mechanism is executed on the node to calculate the attention coefficient e ij (Formula 9) shows the importance of the feature of node i to node j. Then, mask attention is performed to introduce the graph structure into the attention mechanism and calculate the feature from node j∈N i The attention coefficient a ij (Formula 10), in order to make the coefficients easier to compare between different nodes, the Softmax function is used for normalization and the LeakyReLU activation function is introduced; finally, after obtaining the normalized attention coefficient (Formula 11), the node's own feature output d' can be updated by a linear combination of adjacent node features through a nonlinear activation function (Formula 12). In order to enhance the module's spatial correlation feature extraction capability, the module consists of L layers of graph attention layers, d i ∈R F×N Represents the output feature vector of each i-th graph attention layer. The graph attention layer outputs the initial fusion feature vector, and finally iterates to the feature vector x L , where x l＝1 =d, the calculation formula is shown in (13).

[0077]

[0078]

[0079]

[0080]

[0081] x l+1 =MLP l ([x l ||d′]) (13)

[0082] Finally, according to the two task branches of the graph structure semantic feature optimization module, cls and offset, a more regular vector result is obtained. The optimization module cls is to optimize the rough building vector result P according to the graph structure semantic feature. boundary After removing unnecessary vector points, we can get The optimization module offset is based on the semantic features of the graph structure to optimize the rough building vector result P boundary The spatial coordinates are corrected and the eigenvalues ​​are scaled to [0,1] using the sigmoid activation function to obtain the offset vector d offset , then through d offset ·2-1 scales the value to [-1,1]. The optimized building edge vector result P is given by and d offset The calculation formula is shown in (16), where the hyperparameter γ represents the confidence value of the offset result.

[0083] d cls =MLP cls (x L ) (14)

[0084] d offset =MLP offset (x L ) (15)

[0085]

[0086] The final optimized building vector coordinate set P is obtained.

[0087] Step 3: Use the training data set to train the high-resolution remote sensing image building vector extraction model based on the graph attention network. Combined with the loss function, the trained high-resolution remote sensing image building vector extraction model based on the graph attention network is obtained. The specific training process is as follows: Figure 8 shown.

[0088] The model is trained using the loss function so that the total loss function l total Minimize. Among them, the weight parameter λ in the objective loss function v =5,λ a =5,λ v =1.0,λ c =0.25,λ o =0.1,λ s =0.1. In addition, during training, the Adam optimization algorithm was used for training optimization, with a maximum number of iterations of 100, parameters β1 = 0.5, β2 = 0.999, and an initial learning rate of 0.0001. The model convergence progress was detected by jointly optimizing the objective loss function. When the model converged, training was stopped.

[0089] Loss function: The loss function of the model contains the building vector optimization module loss l based on graph attention cls , shape constraint loss l shapeand multi-task segmentation model loss, among which the multi-task segmentation model is composed of building vertex loss l ver , building edge vector angle direction loss l angle and building segmentation mask loss l mask Composition. Total loss l total The calculation is shown in formula (17). In order to balance the difference in magnitude between different loss functions, the loss function weight coefficient is introduced to make the loss change smoother. c ,λ s ,λ v ,λ a and λ m is the weight coefficient of the loss function corresponding to each module.

[0090] l total =λ c ·l cls +λ s ·l shape +λ v ·l ver +λ a ·l angle +λ m ·l mask (17)

[0091] l that constitutes the loss of the multi-task segmentation model mask 、l angle and l ver is the building segmentation mask O obtained by multi-task prediction mask , building edge vector angle direction O angle and building vertex O ver The corresponding sample labels are obtained through cross entropy loss, and the calculation formulas are (18), (19) and (20) respectively.

[0092]

[0093]

[0094]

[0095] The building vector optimization module based on graph attention includes two task branches: optimization module cls and optimization module offset. The optimization module cls selects the result d according to the predicted vector points. cls The binary cross entropy loss is calculated with the sample label, and the calculation formula is shown in (21). The optimization module offset does not perform separate supervised learning, and uses the shape constraint loss function for training.

[0096]

[0097] According to the above modules, the building edge vector result P is obtained. In order to evaluate the vector result and the true label P from the perspective of vector discrete data gt The difference between them can better constrain the simplified shape and construct a vector shape constraint loss function for the final vector P. The shape constraint loss function includes the distance metric loss function l cd and the sinusoidal loss function l sine , calculate the vector shape constraint loss function l shape , as shown in formula (22).

[0098] l shape =l cd +l sine (twenty two)

[0099] Among them, the distance metric loss function is to constrain the distance between the position coordinates and the label, and to encourage the vector coordinates predicted by the model to be closer to the label. This invention draws on the point cloud distance metric loss function in the 3D point cloud field. cd To evaluate the predicted vector result P and the true label P gt The distance difference is shown in formula (23).

[0100]

[0101] Distance metric loss function l cd The building edge vector can be driven to be close to the real building edge. However, for line segments of different lengths, vector points with the same deviation may result in different degrees of shape differences. For short line segments, small deviations of vertices may result in small changes in distance but may lead to large shape changes. The distance metric loss function is not sensitive enough to small deviations, so the sine loss is introduced as an additional angle constraint on the building edge to improve the model's ability to perceive small deviations and encourage the predicted building edge vector to remain parallel to the matching line segment of the real label vector. Sine loss l sine Calculate the predicted vector result P and the true label P gt The sine value difference of the tangent direction of the overall vector point, when l sine The smaller the value, the closer the predicted vector result P is to the actual label P gt The closer to parallelism, the calculation formula is shown in (24), where l represents the line segment formed by a single building instance in the building edge vector result.

[0102]

[0103] 4. Model Usage: After the model is trained, the remote sensing image is input into the trained high-resolution remote sensing image building vector extraction model based on the graph attention network to generate the corresponding building vector outline data.

[0104] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for extracting building vector outline data from high-resolution remote sensing images, characterized in that: The following steps are involved: Step 1: Obtain the original remote sensing image and the corresponding vector contour label data to form a training data set, and then generate the corresponding mask label, vertex label and angle label according to the vector label data; Step 2: Establish a high-resolution remote sensing image building vector contour extraction model based on a graph attention network, select training data from the training dataset, and input the training data and the corresponding mask labels, vertex labels, and angle labels into the high-resolution remote sensing image building vector contour extraction model based on a graph attention network; The high-resolution remote sensing image building vector contour extraction model based on the graph attention network includes a multi-task segmentation model, a rough building edge vector contour construction module and a building vector contour optimization module based on graph attention. The multi-task segmentation model is used to simultaneously obtain the building edge vector angle direction features according to the building's mask label, vertex label and angle label. angle , building mask feature O mask and building vertex distribution characteristics O ver The rough building edge vector outline construction module is used to adopt a method based on edge vector vertex and angle direction sampling, according to the building edge vector angle direction feature O angle , building mask feature O mask and building vertex distribution characteristics O ver Get the rough building vector outline result P boundary The building vector outline optimization module based on graph attention is used to optimize the rough building vector outline result P boundary Optimize and remove redundant coordinate points to make the vector results more regular, and obtain the final optimized building vector outline coordinate set P; Step 3: Based on the obtained building vector outline coordinate set P and training labels, the loss value is calculated using the loss function. The high-resolution remote sensing image building vector outline extraction model based on the graph attention network is trained by repeating step 2, and the model parameters are adjusted until the loss function is minimized. The model training is completed and the final high-resolution remote sensing image building vector outline extraction model based on the graph attention network is obtained. Step 4: Extract building vectors from remote sensing images based on the final high-resolution remote sensing image building vector contour extraction model based on the graph attention network.

2. The method for extracting building vector outline data from high-resolution remote sensing images according to claim 1, wherein: In step 1, generating the angle tag specifically refers to calculating the current position P (x,y) The angle θ∈[-π,π] formed by the vector pointing to the vertex of the building and the x-axis direction is calculated as follows: Step 1.

1. Obtain an angle label matrix with the same resolution as the mask label. Each value in the angle label matrix represents the angle label corresponding to the vector point with the row and column values ​​as coordinates. Step 1.2: Match each vector point in the angle label matrix with a vertex that is closest to it, and inherit the next vertex pointed to by the vertex as the direction of the vector point.

3. The method for extracting building vector outline data from high-resolution remote sensing images according to claim 1, wherein: In step 2, the multi-task segmentation model uses the backbone network Res-U-Net to extract the shared feature map of the image, and then uses three task branches with 256 convolutional layers and ReLU activation functions to extract high-dimensional semantic features in the shared feature map. The three task branches are used to learn the building vertex distribution features, building mask features and building edge vector angle direction features, respectively.

4. The method for extracting building vector outline data from high-resolution remote sensing images according to claim 1, wherein: In step 2, the ECA module is introduced to enhance the feature representation of building edge angles and building vertices, and finally the 1x1 convolution channel number is compressed to obtain the building edge vector angle direction feature. angle , building mask feature O mask and building vertex distribution characteristics O ver .

5. The method for extracting building vector outline data from high-resolution remote sensing images according to claim 1, wherein: In step 2, the method based on edge vector vertex and angle direction sampling includes the following steps: Step 2.1a: Use the NMS algorithm to remove vector points that are too close, and then distribute the features of the building vertices. ver Use a sliding window of size S and step size p to sample the building vertex distribution and retain the vertex with the highest probability in the sliding window; Step 2.2a: Sort the building vertex distribution after NMS algorithm filtering by probability, and select the first N vertices as the rough building edge vector contour points P. boundary ; Step 2.3a: Use the Hungarian algorithm to obtain the connection order of the coordinate points through the distance cost matrix, and finally obtain the rough edge vector contour point P containing the connection order. boundary .

6. The method for extracting building vector outline data from high-resolution remote sensing images according to claim 5, characterized in that: In step 2.3a, the distance cost matrix is ​​composed of the rough building edge vector contour points P boundary The sine distance between each other is constructed using the Hungarian algorithm. The distance cost matrix is ​​regarded as the cost matrix of task allocation. When the overall cost is minimized, the order of the building vector contour points can be obtained. The relationship between the two corresponds to avoid repeated connections. Finally, the rough edge vector contour points P containing the connection order are obtained. boundary .

7. The method for extracting building vector outline data from high-resolution remote sensing images according to claim 1, characterized in that: The working process of the graph attention-based building vector optimization module includes the following steps: Step 2.1b: First, use the rough building vector outline result P boundary The graph structure data of the building edge vector is constructed by the corresponding convolution semantic features, where the graph structure feature vector d is fused with the feature vectors corresponding to the edge vectors in the res2 layer and out layer of U-Res-Net and P boundary Coordinate information of Step 2.2b: The graph attention network uses multiple layers of graph attention layers to extract graph structure semantic features from graph structure data. Step 2.3b, according to the semantic features of the graph structure, the optimization module cls and offset are used to obtain a more regular vector result. The optimization module cls is used to optimize the rough building vector result P according to the semantic features of the graph structure. boundary After removing unnecessary vector points, we can get The optimization module offset is used to optimize the rough building vector result P according to the semantic features of the graph structure. boundary The spatial coordinates are corrected and the eigenvalues ​​are scaled to [0,1] using the sigmoid activation function to obtain the offset vector d offset , then through d offset 2-1 scales the value to [-1,1], and the optimized building edge vector P is given by and d offset Calculated, The hyperparameter γ represents the confidence value of the offset result.

8. The method for extracting building vector outline data from high-resolution remote sensing images according to claim 1, wherein: In step 3, the loss function of the model includes the building vector optimization module loss l based on graph attention. cls , shape constraint loss l shape and multi-task segmentation model loss; where the multi-task segmentation model consists of the building vertex loss l ver , building edge vector angle direction loss l angle and building segmentation mask loss l mask Composition, total loss l total The calculation formula is as follows: l total =λ c ·l cls +λ s ·l shape +λ v ·l ver +λ a ·l angle +λ m ·l mask where λ c ,λ s ,λ v ,λ a and λ m is the weight coefficient of the loss function corresponding to each module.

9. The method for extracting building vector outline data from high-resolution remote sensing images according to claim 8, characterized in that: In step 3, the loss of the multi-task segmentation model is l mask 、l angle and l ver is the building segmentation mask O obtained by multi-task prediction mask , building edge vector angle direction O angle and building vertex O ver The corresponding sample labels are obtained through cross entropy loss. The building vector optimization module based on graph attention includes two task branches: optimization module cls and optimization module offset. The optimization module cls selects the result d according to the predicted vector points. cls The binary cross entropy loss is calculated with the sample label. The optimization module offset does not perform separate supervised learning and uses the shape constraint loss function l shape To train, the shape constraint loss function l shape Contains the distance metric loss function l cd and the sinusoidal loss function l sine , that is: l shape =l cd +l sine .

Images