A method, device, equipment and medium for extracting and regularizing building outlines based on satellite images

Abstract

This application discloses a method, apparatus, device, and medium for building contour extraction and regularization based on satellite imagery. The method includes constructing a base model using a first initial building extraction model based on YOLOv8 and a labeled training dataset; constructing pseudo-labels for unlabeled training data using weak enhancement and the base model via a double-threshold method; extracting a second predicted contour using a second building extraction model with added feature perturbations to the first initial building extraction model; training the building extraction model based on the pseudo-labels and the second predicted contour; and finally extracting the building contour using the building extraction model. This improves the accuracy of GF-7 building extraction even with labeled data, achieving efficient and low-cost building extraction from GF-7 imagery. Simultaneously, the first initial building extraction model based on YOLOv8 incorporates scale feature fusion optimization and edge feature enhancement, effectively improving the accuracy of instance segmentation tasks.

Description

Technical Field

[0001] This application relates to the field of remote sensing image processing technology, and in particular to a method, apparatus, equipment and medium for extracting and regularizing building outlines based on satellite imagery. Background Technology

[0002] Buildings are a core component of urban spatial structure, and their high-precision automatic extraction is of great significance for urban planning, urban renewal, and land use analysis. Urban building extraction is largely based on remote sensing technology. Among them, the Gaofen-7 (GF-7) multi-view stereo satellite, with its sub-meter-level mapping capabilities, has become an important means of building extraction due to its wide coverage, high timeliness, and adaptability to complex environments. However, the current lack of high-quality GF-7 satellite imagery annotation data, and the differences between GF-7 satellite imagery and existing public datasets in spectral features, spatial resolution, and texture features, make it impossible to directly train high-quality building extraction models using GF-7 satellite imagery annotation data. Furthermore, pre-trained models trained on existing public datasets have limited generalization ability and cannot be directly transferred to GF-7 satellite imagery, thus hindering the accurate extraction of buildings from GF-7 satellite imagery using deep learning.

[0003] Therefore, existing technologies still need to be improved and enhanced. Summary of the Invention

[0004] The technical problem to be solved by this application is to provide a method, apparatus, equipment and medium for building outline extraction and regularization based on satellite imagery, which addresses the shortcomings of the existing technology.

[0005] To address the aforementioned technical problems, the first aspect of this application provides a method for building contour extraction and regularization based on satellite imagery, wherein the method specifically includes:

[0006] The first initial building extraction model was trained using the labeled training dataset to obtain the base model;

[0007] Weak and strong augmentation were performed on the unlabeled training data in the unlabeled training dataset to obtain the first and second training data corresponding to the unlabeled training data.

[0008] The first predicted contour, classification confidence, and mask prediction score of the first training data are extracted using the base model, and the first predicted contour is filtered based on the classification confidence and the mask prediction score to obtain pseudo-labels for the first training data.

[0009] The second predicted contour of the second training data is extracted using the second initial building extraction model, wherein the second initial building extraction model is obtained by adding a feature perturbation module to the first initial building extraction model;

[0010] The second initial building extraction model is trained based on the second predicted contour and the pseudo-label to obtain the building extraction model;

[0011] Acquire satellite imagery of the building and use the building extraction model to extract the building outline from the satellite imagery.

[0012] The method for extracting and regularizing building outlines based on satellite imagery includes a first initial building extraction model comprising a backbone network module, a neck module, and a head module, wherein the backbone network module is connected to the neck module; the neck module comprises an upsampling unit, a multi-scale feature fusion unit, and an edge feature unit, wherein the upsampling unit comprises several cascaded upsampling layers, and each subsequent preset number of upsampling layers is connected to the multi-scale feature fusion unit, which is connected to the head module; in the subsequent preset number of upsampling layers, the preceding upsampling layer is connected to the following upsampling layer through an edge feature unit, the last upsampling layer is connected to the head module through an edge feature unit, and the preceding preset number of upsampling layers of the last upsampling layer are connected to the head module.

[0013] The method for extracting and regularizing building outlines based on satellite imagery includes an edge feature unit comprising an edge feature extraction block, a stitching block, a channel attention block, a normalization block, a convolution block, and a fusion block. The edge feature extraction block, the stitching block, the channel attention block, the normalization block, and the fusion block are cascaded sequentially. The convolution block is connected to the fusion block. The inputs of the convolution block and the edge feature extraction block are both inputs of the edge feature unit. The edge feature extraction block is used to perform convolution operations on the inputs in the spatial dimension using the Prewitt operator to extract edge features from the inputs.

[0014] The aforementioned method for extracting and regularizing building contours based on satellite imagery, wherein the step of filtering the first predicted contours based on the classification confidence score and the mask prediction score to obtain pseudo-labels for the first training data specifically includes:

[0015] The classification confidence of the predicted contour in the first predicted contour is compared with the confidence threshold, and the mask prediction score of the predicted contour is compared with the mask score threshold simultaneously.

[0016] If the classification confidence is greater than the confidence threshold and the mask prediction score is greater than the mask score threshold, then the predicted contour is retained to obtain the pseudo-label of the first training data.

[0017] The method for extracting and regularizing building outlines based on satellite imagery includes a weak enhancement method that includes geometric deformation and / or color perturbation, and a strong enhancement method that includes geometric transformation, color transformation, and / or partial occlusion.

[0018] The aforementioned method for building contour extraction and regularization based on satellite imagery, wherein after acquiring satellite imagery of the building and extracting the building contour from the satellite imagery using the building extraction model, the method further includes:

[0019] Extract the boundary lines of connected regions in the building outline to form a closed sequence of boundary points;

[0020] Key points representing structural transitions are selected from the boundary point sequence, and polygonal contours are generated from the selected key points and regularized.

[0021] The aforementioned method for extracting and regularizing building contours based on satellite imagery, wherein the regularization of the selected key points into polygonal contours specifically includes:

[0022] Draw contour lines based on the obtained key points, and calculate the direction angle of each contour line segment;

[0023] The direction angles are quantized using a preset main direction sequence to merge the direction angles into the nearest main direction;

[0024] Based on the quantized direction angle, the contour line segments are geometrically reconstructed, and a polygonal contour is generated based on the reconstructed contour lines.

[0025] The second aspect of this application provides a device for extracting and regularizing building outlines based on satellite imagery, wherein the device specifically includes:

[0026] The pre-training module is used to train the first initial building extraction model using the labeled training dataset to obtain the base model;

[0027] The enhancement module is used to perform weak and strong enhancement on the unlabeled training data in the unlabeled training dataset to obtain the first and second training data corresponding to the unlabeled training data.

[0028] The pseudo-label extraction module is used to extract the first predicted contour, classification confidence and mask prediction score of the first training data using the basic model, and to filter the first predicted contour based on the classification confidence and the mask prediction score to obtain pseudo-labels for the first training data.

[0029] The predicted contour extraction module is used to extract the second predicted contour of the second training data using the second initial building extraction model, wherein the second initial building extraction model is obtained by adding a feature perturbation module to the first initial building extraction model;

[0030] The training module is used to train the second initial building extraction model based on the second predicted contour and the pseudo-label to obtain the building extraction model;

[0031] The building outline extraction module is used to acquire satellite images of buildings and extract the building outlines from the satellite images using the building extraction model.

[0032] A third aspect of this application provides a computer-readable storage medium storing one or more programs that can be executed by one or more processors to implement the steps in the satellite imagery-based building outline extraction and regularization method as described above.

[0033] A fourth aspect of this application provides a terminal device, which includes: a processor and a memory;

[0034] The memory stores a computer-readable program that can be executed by the processor;

[0035] When the processor executes the computer-readable program, it implements the steps in the above-described method for extracting and regularizing building outlines based on satellite imagery.

[0036] Beneficial effects:

[0037] (1) This application uses classification confidence and mask prediction score to screen pseudo-labels and combines pseudo-labels with input-feature dual-space perturbation regularization method. It uses semi-supervised learning that combines a small amount of labeled data and a large amount of unlabeled data to train the building extraction model, which effectively improves the building extraction performance of GF-7 image building extraction model on a small number of labeled samples.

[0038] (2) This application adds a multi-scale feature fusion unit and an edge feature unit to the building extraction model. The multi-scale feature fusion unit introduces low-level features to participate in the fusion, enhances spatial detail information, improves the building extraction model's ability to perceive building edges and small targets, and reduces computational overhead and long-path information transmission loss, thereby improving feature representation ability. The edge feature unit uses edge features as weights on the feature map, making the building contour extraction mode more focused on the target area, suppressing background interference, and improving the clarity and detection accuracy of the building contour.

[0039] (3) The outer boundary of the building is extracted from the building outline obtained in this application and expressed as a closed polygon outline in the form of a point sequence, filtering out small noisy targets and empty areas inside the targets. Then, the closed polygon outline is simplified to refine the high-density boundary points into a small number of key points, and a polyline approximation structure is formed based on the key points to complete the building outline simplification and polygon fitting, further improving the accuracy of the extracted buildings. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 A flowchart of a method for extracting and regularizing building outlines based on satellite imagery provided in an embodiment of this application.

[0042] Figure 2 A flowchart illustrating the training process for extracting models from buildings.

[0043] Figure 3 Extract the network structure diagram of the building model.

[0044] Figure 4 This is a structural diagram of the downsampling unit.

[0045] Figure 5 This is a structural diagram of a multi-scale feature fusion unit.

[0046] Figure 6 This is a structural diagram of edge feature units.

[0047] Figure 7 This is a schematic diagram comparing the image before and after strong enhancement.

[0048] Figure 8 A flowchart illustrating the process of identifying pseudo-labels.

[0049] Figure 9 This is a schematic diagram of regularization.

[0050] Figure 10 A schematic diagram of the building outline extraction and regularization device based on satellite imagery provided in the embodiments of this application.

[0051] Figure 11 A schematic block diagram of the terminal device provided in the embodiments of this application. Detailed Implementation

[0052] This application provides a method, apparatus, device, and medium for building outline extraction and regularization based on satellite imagery. To make the objectives, technical solutions, and effects of this application clearer and more explicit, the following detailed description is provided with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only for explaining this application and are not intended to limit this application.

[0053] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this application means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any units and all combinations of one or more associated listed items.

[0054] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0055] It should be understood that the sequence number and size of each step in this embodiment do not imply the order of execution. The execution order of each process is determined by its function and internal logic, and should not constitute any limitation on the implementation process of this application embodiment.

[0056] The application content will be further explained below with reference to the accompanying drawings and the description of the embodiments.

[0057] This embodiment provides a method for building contour extraction and regularization based on satellite imagery. This method can be used for building contour extraction from domestic high-resolution satellite remote sensing data such as GF-7. This application achieves high-resolution building extraction results through pseudo-label construction, input-feature dual-space perturbation regularization, and a semi-supervised deep learning module. Figure 1 and Figure 2 As shown, the method for extracting and regularizing building outlines based on satellite imagery specifically includes:

[0058] S10. Train the first initial building extraction model using the labeled training dataset to obtain the base model.

[0059] Specifically, the original data in the labeled training dataset and the unlabeled training dataset can be Gaofen-7 satellite image data. The labeled training dataset includes a number of training data with real labels, and the unlabeled training dataset includes a number of training data without real labels. The amount of training data in the unlabeled training dataset is greater than the amount of training data in the labeled training dataset. For example, the labeled training dataset includes 500 training data images, and the unlabeled training dataset includes 5000 training data images.

[0060] Based on this, the process of obtaining the labeled training dataset and the unlabeled training dataset can be as follows: first, acquire a large amount of Gaofen-7 satellite imagery data; then, select a portion of the acquired Gaofen-7 satellite imagery data for manual annotation to obtain the labeled training dataset; and finally, construct the unlabeled training dataset based on the unselected Gaofen-7 satellite imagery data. For example, 10% of the acquired Gaofen-7 satellite imagery data can be selected for annotation to construct the labeled training dataset, and the set of unselected Gaofen-7 satellite imagery data can be used as the unlabeled training dataset; alternatively, a first preset number of Gaofen-7 satellite imagery data can be selected for annotation to construct the labeled training dataset, and a second preset number of Gaofen-7 satellite imagery data can be selected from the remaining Gaofen-7 satellite imagery data to construct the labeled training dataset, and so on.

[0061] It should be noted that, in order to easily distinguish between the training data in the labeled training dataset and the training data in the unlabeled training dataset, the training data in the labeled training dataset is referred to as labeled training data, and the training data in the unlabeled training dataset is referred to as unlabeled training data.

[0062] Furthermore, after obtaining the labeled training dataset, the samples in this dataset can be used for model training. In this embodiment, a first initial building model is selected to train the labeled training data in the labeled training dataset to obtain the basic network used for building extraction. During model training, the Adam algorithm can be used with an initial learning rate set to 0.0001, while learning rate reduction and early stopping methods are employed to quickly obtain the basic network.

[0063] Furthermore, the YOLOv8 network is currently widely used in building extraction models from remote sensing images, as it exhibits excellent performance in both real-time processing and detection. However, the YOLOv8 network still has some limitations in building target detection and instance segmentation tasks, mainly including the following:

[0064] 1) Insufficient multi-scale feature fusion: The YOLOv8 network uses the FPN+PAN structure for multi-scale feature fusion, but ignores the spatial details of low-level features, resulting in insufficient ability to extract building edge information and small target instances.

[0065] 2) The lengthy feature propagation path leads to information degradation and high computational cost: When the PAN structure propagates high-level spatial details from the lower layers, some details are easily lost after multiple levels of convolution and sampling, and the computational complexity increases.

[0066] 3) Background noise interference feature representation: The feature map after fusing low-level information contains a lot of background information of non-building areas (such as vegetation and roads), which makes the model susceptible to background noise interference during instance segmentation, thus causing false detection or blurred building outlines and other problems.

[0067] Since this embodiment utilizes semi-supervised learning to construct the building extraction model, and employs the YOLOv8 network as both the pseudo-label extraction model (i.e., the base model) and the feature extraction model (i.e., the first initial building extraction model) in the semi-supervised learning process, the erroneous information generated by the YOLOv8 network in building object detection and instance segmentation tasks is continuously amplified through the self-reinforcing mechanism of the pseudo-labels, leading to a decline in the extraction performance of the entire semi-supervised learning method. Therefore, the first initial building model provided in this embodiment adopts a YOLOv8-based feature enhancement model, which extends the fusion range of this YOLOv8-based feature enhancement model to feature maps P1-P5, and introduces multi-scale feature fusion units and edge feature units to reduce computational load.

[0068] Specifically, the first initial building extraction model includes a backbone network module, a neck module, and a head module, with the backbone network module connected to the neck module. The neck module includes an upsampling unit, a multi-scale feature fusion unit, and an edge feature unit. The upsampling unit comprises several cascaded upsampling layers, with each subsequent preset number of upsampling layers connected to the multi-scale feature fusion unit, which is connected to the head module. In the subsequent preset number of upsampling layers, the preceding upsampling layer is connected to the following upsampling layer via an edge feature unit, and the final upsampling layer is connected to the head module via an edge feature unit. The preceding preset number of upsampling layers in the final upsampling layer are also connected to the head module. Furthermore, as... Figure 3 As shown, the backbone network module can adopt the backbone network module in the YOLOv8 network, which includes CSB, C2F and SEEP; the head module includes three classification heads.

[0069] It should be noted that the number of downsampling layers included in the upsampling unit and the preset number can be set according to actual needs. In this embodiment, the upsampling unit includes 5 upsampling layers, and the preset number is 3. That is, the backbone network model extracts 5 feature maps, which are labeled as P1-P5 feature maps in descending order of image size. The upsampling unit fuses the P1-P5 feature maps, and the multi-scale feature fusion unit is used to fuse the feature maps extracted by the last three upsampling layers (i.e., the P1 feature map, the P2 feature map, and the P3 feature map).

[0070] In the embodiments of this application, such as Figure 4 As shown, the upsampling unit fuses all feature maps extracted by the backbone network model, avoiding the problem that the YOLOv8 network ignores the edge, texture and spatial location information contained in the P1 and P2 feature maps due to feature fusion only of the P3-P5 feature maps. It captures high-resolution details and spatial information, and improves the overall accuracy of target extraction and small target detection capability.

[0071] like Figure 5As shown, the Triple Feature Encoder Model (TFE) includes a first processing branch, a second processing branch, a third processing branch, and a fusion layer. The first processing branch downsamples the P1 feature map using the AvgPooling module, ensuring that the image size of the downsampled PI feature map is equal to the image size of the P2 feature map, thus preserving high-resolution features and preventing the loss of small target information. Then, the downsampled PI feature map is convolved using the CBS module to enhance the model's nonlinearity and complexity. The second processing branch performs a 1×1 convolution operation on the P2 feature map to maintain model complexity. The second processing branch upsamples the P3 feature map using nearest-neighbor interpolation, ensuring that the image size of the upsampled P3 feature map is equal to the image size of the P2 feature map, thus preserving the richness of local features in the low-resolution image. Then, a 1×1 convolution operation is performed on the upsampled P3 feature map to maintain model complexity. The fusion layer is used to concatenate the feature maps output by the first, second, and third processing branches in the spatial dimension along the channel dimension. Of course, in practical applications, the multi-scale feature fusion unit can also adopt other structures, such as not performing convolution operations on the first, second, and / or third processing branches.

[0072] This embodiment deploys a multi-scale feature fusion unit in the neck module. This unit directly convolves, upsamples, or downsamples the P1-P3 feature maps before concatenating them, avoiding the computational overhead of layer-by-layer upsampling and multiple feature transformations, thus improving the model's inference efficiency. It preserves the high-resolution features of P1 and P2, and downsampling through AvgPooling avoids the loss of detail information that might occur with multiple convolutional downsampling. Direct concatenation in the spatial dimension allows for more direct fusion of low-level detail information and high-level semantic information, avoiding the loss of information during long-path transmission and enhancing the effectiveness of feature transfer.

[0073] Furthermore, since the neck module integrates both P1 and P2 feature maps, while these maps provide more detailed information, they also contain a significant amount of background information. This reduces the model's focusing ability, decreasing its attention to the target region and causing the model to incorrectly focus on non-target areas, thus affecting detection accuracy. Therefore, this embodiment incorporates an edge feature unit in the neck module. This unit extracts salient targets from the feature maps, enabling the model to focus more on the target region and separate it from the background. The edge feature unit can utilize edge operators for feature edge detection, such as the Prewitt operator, Laplacian operator, and Sobel operator.

[0074] In this embodiment, the Prewitt operator is selected as the edge operator used for the edge feature unit. The Prewitt operator is an edge detection method based on first-order differentiation. It detects edges by utilizing the extreme values ​​of gray-level differences between the upper and lower, left and right neighbors of a pixel at the edge, thus removing some false edges. The Prewitt operator uses convolution operations to calculate the gradient of the image in the x-direction (horizontal) and y-direction (vertical), and determines the edge position based on the gradient magnitude. The calculation steps of the Prewitt operator can be as follows:

[0075] 1) Set the convolution kernel in the x-direction ( ):

[0076] = .

[0077] 2) Set the convolution kernel in the y-direction ( )

[0078] = .

[0079] 3) Calculate the horizontal gradient using convolution kernels in the x and y directions. and vertical gradient , where the horizontal gradient and vertical gradient The calculation formulas are:

[0080] ,

[0081] ,

[0082] in, Represents the gradient in the horizontal direction. Represents the gradient in the vertical direction. Represents the convolution kernel in the x-direction. This represents the convolution kernel in the y-direction. Representing an image The pixels in.

[0083] 4) Utilizing horizontal gradients and vertical gradient Calculate the gradient for each pixel, where the formula for calculating the pixel gradient is:

[0084] ,

[0085] in, This represents the gradient of a pixel.

[0086] Based on this, the edge feature unit (denoted as Prewitt) in this embodiment combines the Prewitt operator, weight learning, and attention weighting to enhance the edge features of buildings in the head module. Specifically, as shown... Figure 6 As shown, the edge feature unit includes an edge feature extraction block, a splicing block, a channel attention block, a normalization block, a convolution block, and a fusion block. The edge feature extraction block, the splicing block, the channel attention block, the normalization block, and the fusion block are cascaded in sequence. The convolution block is connected to the fusion block. The input terms of the convolution block and the edge feature extraction block are both input terms of the edge feature unit. The edge feature extraction block is used to perform convolution operations on the input terms in the spatial dimension using the Prewitt operator to extract edge features from the input terms.

[0087] The edge feature map extraction module is used to perform convolution operations on the feature map using the Prewitt operator in the spatial dimension to extract the edge features from the feature map. , This represents the channel dimension. An additional learnable parameter is also set. (Initial value 0.001), used to dynamically adjust edge features in each channel during training. The weights allow the network to adaptively learn the importance of edge information.

[0088] The stitching block is used to stitch together edge features from different channels along the channel dimension to obtain a rich edge representation.

[0089] Channel attention blocks are used to adaptively assign different channel weights and weight them in the channel dimension based on the channel weights. This allows the model to focus on more useful edge features and suppress redundant or irrelevant information, while also enhancing significant edge information and reducing noise interference. This achieves the goal of highlighting key edge information while reducing background noise.

[0090] The normalization block uses the Sigmoid function for non-linear normalization, normalizing the edge weights to a certain value using the Sigmoid function. Within a certain range, this makes the edge weights smoother and avoids abrupt changes. At the same time, it ensures that the weight values ​​are within a reasonable range, improving the stability of the model.

[0091] Convolutional blocks can use the ConvBNSiLU module, which increases the non-linearity of the model, making the network more expressive during the learning process and avoiding gradient vanishing or overfitting during network training.

[0092] The fusion block is used to weight the feature map using edge weights. Specifically, the output of the normalization block is applied to the output of the convolutional block, so that regions with high edge weight values ​​(such as target regions) are enhanced, while regions with low edge weight values ​​(such as background regions) are suppressed. The fusion process of the fusion block can be represented as follows:

[0093] ,

[0094] in, This represents the output term of the normalized block (i.e., the edge weight value). This represents the output of the convolution block (i.e., the feature map after convolution). This represents the output item of the edge feature unit.

[0095] In this embodiment, the edge feature unit uses an edge feature weighting method similar to an attention mechanism to enhance the edge features of buildings, allowing the model to learn to focus on the prominent areas of the target, reducing background interference, and improving the building detection accuracy and optimizing the instance segmentation effect while ensuring lightweight design.

[0096] It should be noted that in practical applications, the initial building extraction model can also adopt other deep learning models according to actual needs. For example, if timeliness is the primary requirement, the YOLOv8 network can be directly used as the initial building extraction model.

[0097] S20. Perform weak and strong augmentation on the unlabeled training data in the unlabeled training dataset to obtain the first and second training data corresponding to the unlabeled training data.

[0098] Specifically, weak and strong augmentation are two different augmentation methods. That is, two different augmentations are applied to the unlabeled training data to obtain the first training data and the second training data. There is a data perturbation between the first and second training data to facilitate training the building extraction model using the first and second training data. This allows the building extraction model to learn a stable representation that does not depend on small data perturbations, reduces its dependence on specific local details, and improves the robustness and generalization performance of the model.

[0099] Weak augmentation preserves the basic structure of the image, while strong augmentation expands the latent feature space of the samples. Optimization reduces the loss between weak and strong augmentation to enhance the model's adaptability and robustness to complex data distributions. Weak augmentation methods can include geometric deformation and / or color perturbation, while strong augmentation methods can include geometric transformation, color transformation, and / or partial region occlusion.

[0100] In one embodiment, weak enhancement can include geometric deformation and color perturbation to preserve the structural information of the image with relatively few transformations. For example, a weak enhancement method might be:

[0101] Random horizontal or vertical flip: Flip horizontally or vertically with a 50% probability.

[0102] Random horizontal or vertical movement: Perform a horizontal or vertical movement with a probability of up to 12.5%.

[0103] Color transformation: Performs HSV image space conversion on the image with a 50% probability.

[0104] In one embodiment, weak enhancement employs a combination of RandAugment and Cutout. Cutout enhances the model's generalization ability by randomly occluding a portion of the image (filling it with zero or mean values). The core idea is to force the model to infer target features from the remaining contextual information by randomly occluding local regions of the image, thereby improving the model's robustness. Furthermore, by constraining the features of different occluded versions, the model's dependence on specific local details is reduced.

[0105] RandAugment uses two hyperparameters, M and N, to control the magnitude of image transformations. M represents the transformation intensity, controlling the magnitude of each data augmentation operation. N represents the number of transformations, controlling the number of augmentation transformations applied to each image. RandAugment reduces the cost of manual adjustments by automatically searching for augmentation strategies, while striking a balance between augmentation diversity and avoiding overfitting. For example, the transformation strategies of RandAugment are shown in Table 1, along with augmented images based on various RandAugment transformation strategies and those based on Cutout. Figure 7 As shown.

[0106] Table 1 Transformation strategies of RandAugment transform

[0107]

[0108] S30. Using the basic model, extract the first predicted contour, classification confidence and mask prediction score of the first training data, and filter the first predicted contour based on the classification confidence and the mask prediction score to obtain the pseudo label of the first training data.

[0109] Specifically, the first training data is the training data obtained through weak augmentation, used to determine the pseudo-labels for the unlabeled data corresponding to the first training data. In other words, the base model is used as the teacher model, and the first training data (i.e., the unlabeled training data after weak augmentation) is used to provide pseudo-labels for the unlabeled training data.

[0110] Classification confidence score represents the reliability of the predicted bounding boxes obtained by the base model and is used as a noise metric. Mask prediction score reflects the mask reliability of the base model prediction and is used as a spatial consistency metric. Both classification confidence score and mask prediction score are used as criteria for filtering pseudo-labels. A dual-threshold strategy is constructed based on classification confidence score and mask prediction score, and this strategy is used to filter the first predicted contours of the base model to improve the reliability of the filtered pseudo-labels.

[0111] In one embodiment, the step of filtering the first predicted contour based on the classification confidence score and the mask prediction score to obtain pseudo-labels for the first training data specifically includes:

[0112] The classification confidence of the predicted contour in the first predicted contour is compared with the confidence threshold, and the mask prediction score of the predicted contour is compared with the mask score threshold simultaneously.

[0113] If the classification confidence is greater than the confidence threshold and the mask prediction score is greater than the mask score threshold, then the predicted contour is retained to obtain the pseudo-label of the first training data.

[0114] Specifically, such as Figure 8 As shown, both the confidence threshold and the mask score threshold are preset. The confidence threshold is used as the filtering criterion for classification confidence, and the mask score threshold is used as the filtering criterion for mask prediction score. When the classification confidence is greater than the confidence threshold, it indicates that the predicted bounding box of the first predicted contour meets the requirements. When the mask prediction score is greater than the mask score threshold, it indicates that the mask corresponding to the first predicted contour meets the requirements. Therefore, the first predicted contour is considered a reliable prediction result. Based on this, the pseudo-label can be represented as:

[0115] ,

[0116] in, Indicates a pseudo tag. Indicates the first predicted profile. Indicates classification confidence level. Indicates the mask score threshold. Indicates the confidence threshold. This represents the mask score threshold.

[0117] This application employs a dual-threshold strategy for pseudo-label screening. It suppresses false alarms due to positioning bias by using classification confidence and suppresses missed targets corresponding to low-quality masks by using mask prediction scores, thereby improving the reliability of pseudo-labels.

[0118] S40. Use the second initial building extraction model to extract the second predicted contour of the second training data;

[0119] Specifically, the second initial building extraction model is obtained by adding a feature perturbation module to the first initial building extraction model. The second initial building extraction model extracts the second predicted contour corresponding to the second training data. In other words, the second initial building extraction model is used as a student model, and the pseudo-labels generated by the base model (i.e., the teacher model) are used as the supervision signal for the second initial building extraction model. The knowledge learned by the base model is transferred to the second initial building extraction model, thereby achieving the training of the second initial building extraction model.

[0120] The second initial building extraction model is obtained by adding a feature perturbation module to the first initial building extraction model. The feature perturbation module is used to add perturbation to the feature map extracted by the first initial building extraction model, so as to realize the synchronous perturbation of the training data (i.e., input perturbation) and the feature map (i.e. feature perturbation) corresponding to the training data. This can further enhance the building extraction model's ability to extract global semantic features and improve generalization.

[0121] The feature perturbation module inserts a channel dropout operation into the feature space. This operation maintains consistency in predictions at the image and feature levels for unlabeled training data, making the input perturbation and feature perturbation complementary. However, in practical applications, because channel dropout only operates on the channel dimension, randomly discarding some feature channels, it fails to explicitly disrupt the spatial continuity of the feature map. In convolutional neural networks, features typically have strong spatial correlations, and the model can still predict based on local regions, thus limiting its ability to model global features.

[0122] Based on this, in one embodiment, the feature perturbation module employs the feature space-based regularization method DropBlock to further enhance the effectiveness of feature perturbation. DropBlock is a regularization method designed to enhance the robustness of the model by structurally discarding continuous regions in the feature map. Its core idea is to force the network to learn a more global feature representation by disrupting the spatial continuity of the feature map.

[0123] like Figure 9 As shown, the DropBlock process involves pre-setting the size of the occluded region, then randomly selecting a center point on the image's feature map, and generating an occluded region (e.g., filled with 0s) based on the size of the occluded region to create a mask image. Finally, this mask image is multiplied by the feature map to obtain a perturbed feature map, which discards the feature information of the occluded region. The calculation process of DropBlock can be summarized as follows:

[0124] 1) Calculate the sampling probability γ of the center point, where the formula for calculating γ is:

[0125] ,

[0126] in, This represents the feature retention probability (similar to the retention probability in Dropout). and This represents the height and width of the feature map. Indicates the size of the occluded area.

[0127] 2) Generate a binary mask, where each position is independently activated with probability (as the center of the discard block), where the binary mask is represented as:

[0128] ,

[0129] in, Representing the Bernoulli distribution, with probability Generate 1 with probability Generate 0, Represents the mask value. This indicates that the selected position is the center of the discard block. This indicates that the location was not selected as the center of the discard block.

[0130] 3) Expand the center point through convolution operation. The center mask block, in which The center mask block is represented as:

[0131] ,

[0132] in, Indicates the center mask block, Represents a binary mask. Indicates the size is All of them have a 1 convolution kernel. This indicates that convolution operations require padding to maintain the size.

[0133] 4) Invert the block mask to obtain the mask image of the reserved region. The mask image is represented as follows:

[0134] ,

[0135] in, This represents a mask image.

[0136] S50. The second initial building extraction model is trained based on the second predicted contour and the pseudo-label to obtain the building extraction model.

[0137] Specifically, after obtaining the second predicted contour and pseudo-labels, a loss function can be constructed based on the second predicted contour and pseudo-labels, and this loss function can be used to train the second initial building extraction model. However, since the first and second training data were obtained through different data augmentations (the first training data has weak augmentation, and the second training data has strong augmentation), the pseudo-labels and the second predicted contours may have spatial inconsistencies. Therefore, before constructing the loss function based on the second predicted contour and pseudo-labels, a spatial alignment mechanism can be used to spatially align the pseudo-labels with the second predicted contours to ensure the mapping relationship between them.

[0138] In one embodiment, the alignment process between the pseudo-label and the second predicted contour can be as follows:

[0139] First, obtain the scaling factor between the weak augmentation corresponding to the first training data and the strong augmentation corresponding to the second training data. Rotation angle , Flip parameters and translation offset And according to the scaling factor Rotation angle , Flip parameters and translation offset Calculate the global affine transformation matrix , where the global affine transformation matrix The calculation formula can be:

[0140] ,

[0141] in, and This indicates the flip parameter.

[0142] Secondly, the pseudo-labels are mapped to the input space of the second initial building extraction model using the global affine transformation matrix, so as to align the pseudo-labels with the second predicted contours. The mapped pseudo-labels are as follows:

[0143] ,

[0144] ,

[0145] in, This represents the prediction box in the pseudo-label. This represents the mask image in the pseudo-tag. This represents the predicted bounding box in the mapped pseudo-label. The mask image in the mapped pseudo-label.

[0146] Finally, after training the second initial building extraction model on the unlabeled training dataset, the feature perturbations in the trained second initial building extraction model are removed to obtain the building extraction model, that is, the building extraction model does not include feature perturbations.

[0147] S60. Acquire satellite imagery of the building and extract the building outline from the satellite imagery using the building extraction model.

[0148] Specifically, building satellite imagery can be obtained from domestically produced Gaofen-7 satellites, and the building extraction model is trained through steps S10-S50. The building satellite imagery serves as input to the building extraction model, which outputs the building outline corresponding to that satellite imagery. Furthermore, after acquiring the building outline, the bounding box (i.e., boundary) of each building instance within that outline can be extracted and preprocessed to further improve the segmentation accuracy of the building instances. Preprocessing may include regularization and / or vectorization.

[0149] In one embodiment, after acquiring satellite imagery of the building and extracting the building outline from the satellite imagery using the building extraction model, the method further includes:

[0150] Extract the boundary lines of connected regions in the building outline to form a closed sequence of boundary points;

[0151] Key points representing structural transitions are selected from the boundary point sequence, and polygonal contours are generated from the selected key points and regularized.

[0152] Specifically, the boundary point sequence is the set of boundary points obtained by extracting the connected regions of the building outline. In other words, when extracting the boundary lines of the connected regions of the building outline, several outline point sequences will be extracted from the building outline, and each outline point sequence is a boundary point sequence of a connected region.

[0153] In one embodiment, the boundary lines of the connected regions are extracted from the binary image corresponding to the building contours using an eight-neighbor contour tracking algorithm. The specific extraction process includes first finding the first foreground pixel (value 255) in the binary image and setting it to... As the starting point, then from Starting from there, search for the next boundary point within its territory in a fixed direction (counter-clockwise). and with Continue searching for neighboring points around the center, and construct boundary sequences in sequence. This process continues until the starting point is closed, resulting in an ordered set of points representing the building boundary, i.e., a sequence of closed boundary points.

[0154] Furthermore, after obtaining the boundary point sequence, key points can be selected from the boundary point sequence, and the boundary contour can be regenerated based on the key points to improve the efficiency of contour expression and structural clarity. These key points include boundary points used to represent structural transitions, and may also include boundary points used for structural transitions and partial boundary points between two connected boundary points used for structural transitions.

[0155] In one embodiment, the Douglas-Peucker algorithm can be used to simplify the building to obtain key points representing structural transitions. The goal of the Douglas-Peucker algorithm is to remove redundant points and retain key points representing structural transitions, generating a polyline (polygon) that approximates the original contour, while ensuring that the overall shape error of the outline does not exceed a given threshold. Specifically, let the sequence of boundary points corresponding to the boundary line of a connected tube region be... The process of selecting key points using the Douglas-Peucker algorithm is as follows:

[0156] 1) Take the first and second boundary points and last boundary point As the initial line segment;

[0157] 2) Calculate the boundary point sequence and The perpendicular distance from the intermediate boundary point to the initial line segment;

[0158] 3) Find the boundary point with the largest vertical distance. Record the vertical distance. ;

[0159] 4) If If the preset error threshold is met, then the boundary point will be retained. The initial line segment is divided into two segments. and Then respectively and As the initial line segment, repeat step 2);

[0160] 5) If This indicates that intermediate boundary points have little impact on the overall shape, so all intermediate boundary points are discarded, thus completing the key point selection.

[0161] After the key points are selected, a polyline profile is constructed based on the selected key points to obtain the polygonal profile corresponding to the building profile. The number of boundary points of the polygonal profile is less than the number of boundary points of the building profile, but the geometry of the polygonal profile is consistent with the geometry of the building profile.

[0162] Furthermore, in remote sensing imagery, especially high-resolution imagery, building outlines often exhibit distinct directional characteristics, such as common orthogonal buildings (0° / 90°) and sloping roof buildings (45° / 135°). However, the building outlines output by deep learning models are pixel-level raster edges. After conversion to polygons, the direction of edge segments may exhibit issues such as: slight angular shifts (e.g., 87°, 91°); unstructured stray directions (e.g., 78.6°, 112.3°); and inconsistent combinations of multiple directions. Therefore, when regularizing the polygonal outlines generated from key points, the direction angles of each line segment within multiple polygonal outlines can be aligned towards the main direction, and the line segments can be regularized to unify the building edge directions and improve the regularity of the outlines.

[0163] For example, the regularization of the polygonal contour generated from the selected key points specifically includes:

[0164] Draw contour lines based on the obtained key points, and calculate the direction angle of each contour line segment;

[0165] The direction angles are quantized using a preset main direction sequence to merge the direction angles into the nearest main direction;

[0166] Based on the quantized direction angle, the contour line segments are geometrically reconstructed, and a polygonal contour is generated based on the reconstructed contour lines.

[0167] Specifically, the direction angle of a contour line segment refers to the angle between the contour line segment and the positive direction of the horizontal axis. The formula for calculating the direction angle of a contour line segment is as follows:

[0168] ,

[0169] in, Indicates the direction angle of the outline segment, ( , ) represents one endpoint of the outline segment, ( , () indicates the other endpoint of the outline segment.

[0170] The preset principal direction sequence is pre-constructed based on the structural characteristics of the building and includes several principal direction angles. For example, for orthogonal buildings, the preset principal direction sequence... For structures with sloping roofs or complex structures, a pre-defined main direction sequence is used. .

[0171] Quantifying the direction angle using a preset principal direction sequence refers to selecting the principal direction whose direction angle is closest to that of the contour line segment from the preset principal direction sequence. The closest principal direction is the principal direction with the smallest angular difference between its direction angle and that of the contour line segment. The closest principal direction can be expressed as: ,in, Indicates the direction angle of the outline line segment The closest principal direction, Indicates the preset main direction sequence. This represents the main direction in the preset main direction sequence. Then, the angle difference between the direction angle closest to the main direction and the contour line segment is calculated. Deviation threshold from preset (like ~ ) compare, if Then the direction angle of the contour line segment is quantized as , Then the direction angle of the outline segment remains unchanged.

[0172] Furthermore, after quantizing the direction angle of each contour line segment, for the contour line segment whose direction angle is quantized to be closest to the main direction, the contour line segment is reconstructed based on the quantized direction angle. The reconstruction process can use the centerline rotation method, retaining the midpoint of the line segment, and rotating the original line segment as a whole to align its direction with the quantized direction.

[0173] In summary, this embodiment provides a method for building contour extraction and regularization based on satellite imagery. This method includes constructing a base model using a first initial building extraction model based on YOLOv8 and a labeled training dataset; constructing pseudo-labels for unlabeled training data using weak enhancement and the base model via a double-threshold method; extracting a second predicted contour using a second building extraction model with added feature perturbations to the first initial building extraction model; training the building extraction model based on the pseudo-labels and the second predicted contour; and finally, extracting the building contour using the building extraction model. This improves the accuracy of GF-7 building extraction even with labeled data, achieving efficient and low-cost building extraction from GF-7 imagery. Furthermore, the first initial building extraction model based on YOLOv8 incorporates scale feature fusion optimization and edge feature enhancement, effectively improving the accuracy of instance segmentation tasks.

[0174] Based on the above-mentioned method for building contour extraction and regularization based on satellite imagery, this embodiment provides a device for building contour extraction and regularization based on satellite imagery, such as... Figure 10 As shown, the satellite imagery-based building contour extraction and regularization device specifically includes:

[0175] The pre-training module 100 is used to train the first initial building extraction model using the labeled training dataset to obtain the base model;

[0176] The enhancement module 200 is used to perform weak and strong enhancement on the unlabeled training data in the unlabeled training dataset to obtain the first training data and the second training data corresponding to the unlabeled training data.

[0177] The pseudo-label extraction module 300 is used to extract the first predicted contour, classification confidence and mask prediction score of the first training data using the basic model, and to filter the first predicted contour based on the classification confidence and the mask prediction score to obtain pseudo-labels of the first training data.

[0178] The prediction contour extraction module 400 is used to extract the second prediction contour of the second training data using the second initial building extraction model, wherein the second initial building extraction model is obtained by adding a feature perturbation module to the first initial building extraction model;

[0179] Training module 500 is used to train the second initial building extraction model based on the second predicted contour and the pseudo-label to obtain the building extraction model;

[0180] The building outline extraction module 600 is used to acquire satellite images of buildings and extract the building outlines of buildings in the satellite images using the building extraction model.

[0181] Based on the above-described method for building outline extraction and regularization based on satellite imagery, this embodiment provides a computer-readable storage medium storing one or more programs that can be executed by one or more processors to implement the steps in the method for building outline extraction and regularization based on satellite imagery as described in the above embodiment.

[0182] Based on the aforementioned method for building outline extraction and regularization using satellite imagery, this application also provides a terminal device, such as... Figure 11 As shown, it includes at least one processor 20; a display screen 21; and a memory 22, and may also include a communications interface 23 and a bus 24. The processor 20, display screen 21, memory 22, and communications interface 23 can communicate with each other via the bus 24. The display screen 21 is configured to display a preset user guide interface in the initial setup mode. The communications interface 23 can transmit information. The processor 20 can invoke logical instructions in the memory 22 to execute the methods described in the above embodiments.

[0183] Furthermore, the logical instructions in the aforementioned memory 22 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium.

[0184] The memory 22, as a computer-readable storage medium, can be configured to store software programs, computer-executable programs, such as program instructions or modules corresponding to the methods in the embodiments of this disclosure. The processor 20 executes functional applications and data processing by running the software programs, instructions, or modules stored in the memory 22, thereby implementing the methods in the above embodiments.

[0185] The memory 22 may include a program storage area and a data storage area. The program storage area may store the operating system and application programs required for at least one function; the data storage area may store data created based on the use of the terminal device. Furthermore, the memory 22 may include high-speed random access memory (RAM) and non-volatile memory. Examples include various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks, as well as transient storage media.

[0186] Furthermore, the specific process of loading and executing multiple instruction processors in the aforementioned storage medium and terminal device has been described in detail in the above method, and will not be repeated here.

[0187] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for building contour extraction and regularization based on satellite imagery, characterized in that, The aforementioned method for building contour extraction and regularization based on satellite imagery specifically includes: The first initial building extraction model was trained using the labeled training dataset to obtain the base model; Weak and strong augmentation were performed on the unlabeled training data in the unlabeled training dataset to obtain the first and second training data corresponding to the unlabeled training data. The first predicted contour, classification confidence, and mask prediction score of the first training data are extracted using the base model, and the first predicted contour is filtered based on the classification confidence and the mask prediction score to obtain pseudo-labels for the first training data. The second predicted contour of the second training data is extracted using the second initial building extraction model, wherein the second initial building extraction model is obtained by adding a feature perturbation module to the first initial building extraction model; The second initial building extraction model is trained based on the second predicted contour and the pseudo-label to obtain the building extraction model; Acquire satellite imagery of the building and use the building extraction model to extract the building outline from the satellite imagery.

2. The method for building contour extraction and regularization based on satellite imagery according to claim 1, characterized in that, The first initial building extraction model includes a backbone network module, a neck module, and a head module. The backbone network module is connected to the neck module. The neck module includes an upsampling unit, a multi-scale feature fusion unit, and an edge feature unit. The upsampling unit includes several cascaded upsampling layers. Each subsequent preset number of upsampling layers is connected to the multi-scale feature fusion unit, which is connected to the head module. In each subsequent preset number of upsampling layers, the preceding upsampling layer is connected to the following upsampling layer through an edge feature unit. The last upsampling layer is connected to the head module through an edge feature unit, and the preceding preset number of upsampling layers of the last upsampling layer are connected to the head module.

3. The method for building contour extraction and regularization based on satellite imagery according to claim 2, characterized in that, The edge feature unit includes an edge feature extraction block, a splicing block, a channel attention block, a normalization block, a convolution block, and a fusion block. The edge feature extraction block, the splicing block, the channel attention block, the normalization block, and the fusion block are cascaded in sequence. The convolution block is connected to the fusion block. The input terms of the convolution block and the edge feature extraction block are both input terms of the edge feature unit. The edge feature extraction block is used to perform convolution operations on the input terms in the spatial dimension using the Prewitt operator to extract edge features from the input terms.

4. The method for building contour extraction and regularization based on satellite imagery according to claim 1, characterized in that, The step of filtering the first predicted contour based on the classification confidence score and the mask prediction score to obtain pseudo-labels for the first training data specifically includes: The classification confidence of the predicted contour in the first predicted contour is compared with the confidence threshold, and the mask prediction score of the predicted contour is compared with the mask score threshold simultaneously. If the classification confidence is greater than the confidence threshold and the mask prediction score is greater than the mask score threshold, then the predicted contour is retained to obtain the pseudo-label of the first training data.

5. The method for building contour extraction and regularization based on satellite imagery according to claim 1, characterized in that, The weak enhancement includes geometric deformation and / or color perturbation, while the strong enhancement includes geometric transformation, color transformation, and / or partial area occlusion.

6. The method for building contour extraction and regularization based on satellite imagery according to claim 1, characterized in that, After acquiring satellite imagery of the building and extracting the building outline from the satellite imagery using the building extraction model, the method further includes: Extract the boundary lines of connected regions in the building outline to form a closed sequence of boundary points; Key points representing structural transitions are selected from the boundary point sequence, and polygonal contours are generated from the selected key points and regularized.

7. The method for building contour extraction and regularization based on satellite imagery according to claim 6, characterized in that, The process of regularizing the polygonal contour generated from the selected key points specifically includes: Draw contour lines based on the obtained key points, and calculate the direction angle of each contour line segment; The direction angles are quantized using a preset main direction sequence to merge the direction angles into the nearest main direction; Based on the quantized direction angle, the contour line segments are geometrically reconstructed, and a polygonal contour is generated based on the reconstructed contour lines.

8. A device for extracting and regularizing building outlines based on satellite imagery, characterized in that, The aforementioned satellite imagery-based building contour extraction and regularization device specifically includes: The pre-training module is used to train the first initial building extraction model using the labeled training dataset to obtain the base model; The enhancement module is used to perform weak and strong enhancement on the unlabeled training data in the unlabeled training dataset to obtain the first and second training data corresponding to the unlabeled training data. The pseudo-label extraction module is used to extract the first predicted contour, classification confidence and mask prediction score of the first training data using the basic model, and to filter the first predicted contour based on the classification confidence and the mask prediction score to obtain pseudo-labels for the first training data. The predicted contour extraction module is used to extract the second predicted contour of the second training data using the second initial building extraction model, wherein the second initial building extraction model is obtained by adding a feature perturbation module to the first initial building extraction model; The training module is used to train the second initial building extraction model based on the second predicted contour and the pseudo-label to obtain the building extraction model; The building outline extraction module is used to acquire satellite images of buildings and extract the building outlines from the satellite images using the building extraction model.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores one or more programs, which can be executed by one or more processors to implement the steps in the satellite imagery-based building outline extraction and regularization method as described in any one of claims 1-7.

10. A terminal device, characterized in that, include: Processor and memory; The memory stores a computer-readable program that can be executed by the processor; When the processor executes the computer-readable program, it implements the steps in the method for building outline extraction and regularization based on satellite imagery as described in any one of claims 1-7.

Images