Building contour extraction and regularization method and device based on satellite image, equipment and medium

Abstract

The invention discloses a building contour extraction and regularization method, device and equipment based on a satellite image and a medium, and the method comprises the steps: constructing a basic model through a first initial building extraction model based on YOLOv8 and an annotation training data set, constructing a pseudo tag for unannotated training data through a dual-threshold method by employing weak enhancement and the basic model, and extracting a second prediction contour by using a second building extraction model in which feature disturbance is added in the first initial building extraction model, training the building extraction model based on a pseudo tag and the second prediction contour, and extracting a building contour by using the building extraction model, so that the GF-7 building extraction precision can be improved under the condition of annotated data. And efficient and low-cost GF-7 image building extraction is realized. And meanwhile, scale feature fusion optimization and edge features are introduced into the first initial building extraction model based on YOLOv8 for feature enhancement, so that the instance segmentation task precision is effectively improved.

Description

Technical Field

[0001] The present application relates to the technical field of remote sensing image processing, and in particular to a method, device, equipment and medium for extracting and regularizing building outlines based on satellite images. Background Art

[0002] Buildings are a core component of urban spatial structure, and their high-precision automatic extraction is of great significance to fields such as urban planning, urban renewal, and land use analysis. Urban building extraction is largely based on remote sensing technology. The Gaofen-7 (GF-7) multi-view stereo satellite, with its sub-meter mapping capabilities, has become a key tool for building extraction due to its wide coverage, high timeliness, and adaptability to complex environments. However, due to the current lack of high-quality annotated GF-7 satellite imagery data and the differences between GF-7 satellite imagery and existing public datasets in terms of spectral characteristics, spatial resolution, and texture characteristics, it is impossible to directly use GF-7 satellite imagery annotated data to train high-quality building extraction models. Furthermore, pre-trained models trained on existing public datasets have limited generalization capabilities and cannot be directly transferred to GF-7 satellite imagery. Consequently, it is impossible to accurately extract buildings from GF-7 satellite imagery using deep learning.

[0003] Therefore existing technology still needs to be improved and improved. Summary of the Invention

[0004] The technical problem to be solved by this application is to provide a method, device, equipment and medium for extracting and regularizing building outlines based on satellite images in response to the shortcomings of the existing technology.

[0005] In order to solve the above technical problems, the first aspect of the present application provides a method for extracting and regularizing building outlines based on satellite images, wherein the method for extracting and regularizing building outlines based on satellite images specifically includes: Training the first initial building extraction model using the labeled training data set to obtain a base model; Performing weak enhancement and strong enhancement on the unlabeled training data in the unlabeled training dataset respectively to obtain first training data and second training data corresponding to the unlabeled training data; Extracting a first prediction profile, a classification confidence, and a mask prediction score of the first training data using the base model, and filtering the first prediction profile based on the classification confidence and the mask prediction score to obtain a pseudo label for the first training data; Extracting a second predicted profile of the second training data using a second initial building extraction model, wherein the second initial building extraction model is obtained by adding a feature perturbation module to the first initial building extraction model; Training the second initial building extraction model according to the second predicted contour and the pseudo label to obtain a building extraction model; A building satellite image is acquired, and building outlines of buildings in the building satellite image are extracted using the building extraction model.

[0006] The satellite image-based building outline extraction and regularization method, wherein the first initial building extraction model includes a backbone network module, a neck module and a head module, the backbone network module is connected to the neck module; the neck module includes an upsampling unit, a multi-scale feature fusion unit and an edge feature unit, the upsampling unit includes several cascaded upsampling layers, the upsampling layers of the subsequent preset number of layers are all connected to the multi-scale feature fusion unit, and the multi-scale feature fusion unit is connected to the head module; the previous upsampling layer of the two adjacent upsampling layers in the subsequent preset number of layers is connected to the next upsampling layer through the edge feature unit, the last upsampling layer is connected to the head module through the edge feature unit, and the upsampling layers of the previous preset number of layers of the final upsampling layer are connected to the head module.

[0007] The satellite image-based building outline extraction and regularization method, wherein the edge feature unit includes an edge feature extraction block, a splicing block, a channel attention block, a normalization block, a convolution block and a fusion block, the edge feature extraction block, the splicing block, the channel attention block, the normalization block and the fusion block are cascaded in sequence, and the convolution block is connected to the fusion block, wherein the input items of the convolution block and the edge feature extraction block are both input items of the edge feature unit, and the edge feature extraction block is used to perform a convolution operation on the input item using a Prewitt operator in the spatial dimension to extract edge features in the input item.

[0008] In the satellite image-based building outline extraction and regularization method, the step of screening the first predicted outline based on the classification confidence and the mask prediction score to obtain a pseudo label for the first training data specifically includes: comparing the classification confidence of the predicted contour in the first predicted contour with a confidence threshold, and simultaneously comparing the mask prediction score of the predicted contour with a mask score threshold; If the classification confidence is greater than the confidence threshold and the mask prediction score is greater than the mask score threshold, the predicted contour is retained to obtain a pseudo label for the first training data.

[0009] In the satellite image-based building outline extraction and regularization method, the weak enhancement includes geometric deformation and / or color disturbance, and the strong enhancement includes geometric transformation, color transformation and / or partial area occlusion.

[0010] The satellite image-based building outline extraction and regularization method, wherein, after acquiring the building satellite image and extracting the building outline of the building in the building satellite image using the building extraction model, the method further comprises: Extracting connected area boundary lines in the building outline to form a closed boundary point sequence; Key points representing structural turning points are selected from the boundary point sequence, and polygonal contours generated from the selected key points are regularized.

[0011] The method for extracting and regularizing building outlines based on satellite images, wherein the step of regularizing polygonal outlines generated from selected key points specifically includes: Draw contour segments based on the obtained key points and calculate the direction angle of each contour segment; quantizing the direction angles using a preset main direction sequence to merge the direction angles into the nearest main direction; The contour line segments are geometrically reconstructed according to the quantized direction angles, and a polygonal contour is generated based on the reconstructed contour lines.

[0012] A second aspect of the present application provides a device for extracting and regularizing building outlines based on satellite images, wherein the device for extracting and regularizing building outlines based on satellite images specifically comprises: A pre-training module, configured to train a first initial building extraction model using a labeled training dataset to obtain a basic model; An enhancement module is used to perform weak enhancement and strong enhancement on the unlabeled training data in the unlabeled training data set, respectively, to obtain first training data and second training data corresponding to the unlabeled training data; a pseudo-label extraction module, configured to extract a first prediction profile, a classification confidence, and a mask prediction score of the first training data using the base model, and filter the first prediction profile based on the classification confidence and the mask prediction score to obtain a pseudo-label for the first training data; a predicted contour extraction module, configured to extract a second predicted contour of the second training data using a second initial building extraction model, wherein the second initial building extraction model is obtained by adding a feature perturbation module to the first initial building extraction model; a training module, configured to train the second initial building extraction model according to the second predicted contour and the pseudo label to obtain a building extraction model; The building outline extraction module is used to obtain a building satellite image and extract the building outline of the building in the building satellite image using the building extraction model.

[0013] The third aspect of the present application provides a computer-readable storage medium, which stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps in any of the satellite image-based building outline extraction and regularization methods described above.

[0014] A fourth aspect of the present application provides a terminal device, comprising: a processor and a memory; The memory stores a computer-readable program executable by the processor; When the processor executes the computer-readable program, the processor implements the steps in any of the above-mentioned methods for extracting and regularizing building outlines based on satellite images.

[0015] Beneficial effects: (1) This application uses classification confidence and mask prediction scores to screen pseudo labels, combines pseudo labels with the regularization method of input-feature dual space perturbation, and uses semi-supervised learning that combines a small amount of labeled data with a large amount of unlabeled data to train the building extraction model, effectively improving the building extraction performance of the building extraction model in GF-7 images on a small amount of labeled samples.

[0016] (2) This application adds a multi-scale feature fusion unit and an edge feature unit to the building extraction model. The multi-scale feature fusion unit introduces the underlying features to participate in the fusion, enhances the spatial detail information, and improves the building extraction model's perception of building edges and small targets. At the same time, it can reduce computational overhead and reduce long-path information transmission loss, thereby improving feature expression capabilities. The edge feature unit uses edge features as weights to act on the feature map, making the building contour extraction mode pay more attention to the target area, suppressing background interference, and improving the clarity and detection accuracy of the building contour.

[0017] (3) The outer boundary of the building is extracted from the building outline extracted by this application and expressed as a closed polygonal outline in the form of a point sequence, filtering out small noise targets and internal void areas of the target. Then, the closed polygonal outline is simplified to refine the high-density boundary points into a small number of key points, and a polyline approximation structure is formed based on the key points to complete the building outline simplification and polygon fitting, further improving the accuracy of the extracted buildings. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0019] Figure 1 Flowchart of the method for extracting and regularizing building outlines based on satellite images provided in an embodiment of the present application.

[0020] Figure 2 Flowchart of the training process for the building extraction model.

[0021] Figure 3 Extract the network structure diagram of the model for the building.

[0022] Figure 4 This is the structural diagram of the downsampling unit.

[0023] Figure 5 This is the structural diagram of the multi-scale feature fusion unit.

[0024] Figure 6 This is the structural diagram of the edge feature unit.

[0025] Figure 7 Schematic diagram of the comparison between the image before and after strong enhancement.

[0026] Figure 8 Flowchart of the pseudo-label determination process.

[0027] Figure 9 Schematic diagram of regularization.

[0028] Figure 10 This is a block diagram of the principles of the satellite image-based building outline extraction and regularization device provided in an embodiment of the present application.

[0029] Figure 11 This is a block diagram of the principles of the terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0030] The embodiments of the present application provide a method, apparatus, device, and medium for extracting and regularizing building outlines based on satellite imagery. To make the objectives, technical solutions, and effects of the present application more clear and explicit, the present application is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are intended only to illustrate the present application and are not intended to limit the present application.

[0031] It will be understood by those skilled in the art that, unless expressly stated otherwise, the singular forms "a", "an", "said" and "the" used herein may also include the plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be intermediate elements. In addition, "connected" or "coupled" as used herein may include wireless connections or wireless couplings. The term "and / or" used herein includes all or any units and all combinations of one or more associated listed items.

[0032] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those skilled in the art to which this application belongs. It should also be understood that terms such as those defined in common dictionaries should be understood to have meanings consistent with their meanings in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0033] It should be understood that the sequence numbers and sizes of the steps in this embodiment do not imply the order of execution. The order of execution of each process is determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of this application.

[0034] The application content will be further explained below through description of embodiments in conjunction with the accompanying drawings.

[0035] This embodiment provides a building outline extraction and regularization method based on satellite images, which can be used to extract building outlines from domestic high-resolution satellite remote sensing data such as GF-7. This application uses pseudo-label construction, input-feature dual space perturbation regularization method, and semi-supervised learning deep learning module building extraction to obtain high-resolution building extraction results. Figure 1 and Figure 2 As shown, the building outline extraction and regularization method based on satellite images specifically includes: S10. Train the first initial building extraction model using the labeled training data set to obtain a basic model.

[0036] Specifically, the original data in the labeled training dataset and the unlabeled training dataset can be Gaofen-7 satellite image data, the labeled training dataset includes a number of training data with real labels, and the unlabeled training dataset includes a number of training data without real labels, and the amount of training data of the unlabeled training dataset is greater than the amount of training data of the labeled training dataset. For example, the labeled training dataset includes 500 training data, and the unlabeled training dataset includes 5,000 training data, etc.

[0037] Based on this, the process of obtaining the labeled training dataset and the unlabeled training dataset can be to first obtain a large amount of Gaofen-7 satellite image data, then select some Gaofen-7 satellite image data from the obtained Gaofen-7 satellite image data for manual labeling to obtain a labeled training dataset, and construct an unlabeled training dataset based on the unselected Gaofen-7 satellite image data. For example, 10% of the acquired Gaofen-7 satellite image data is selected for labeling to construct a labeled training dataset, and the set of unselected Gaofen-7 satellite image data is used as the unlabeled training dataset; or, a first preset number of Gaofen-7 satellite image data is selected from the acquired Gaofen-7 satellite image data for labeling to construct a labeled training dataset, and a second preset number of the remaining Gaofen-7 satellite image data is selected to construct a labeled training dataset, etc.

[0038] It should be noted that, in order to facilitate the distinction between the training data in the labeled training dataset and the training data in the unlabeled training dataset, the training data in the labeled training dataset is recorded as labeled training data, and the training data in the unlabeled training dataset is recorded as unlabeled training data.

[0039] Furthermore, after obtaining the labeled training dataset, the samples in the sample set can be used for model training. In the embodiment of the present application, the first initial building model is selected to train the labeled training data in the labeled training dataset to obtain a basic network for extracting buildings. During the model training process, the Adam algorithm can be used with an initial learning rate set to 0.0001. Learning rate reduction and early stopping are also used to quickly obtain the basic network.

[0040] Furthermore, the YOLOv8 network is currently widely used in building extraction models for remote sensing images. The YOLOv8 network performs well in both real-time and detection performance. However, the YOLOv8 network still has some problems in building object detection and instance segmentation tasks, mainly including the following: 1) Insufficient multi-scale feature fusion: The YOLOv8 network uses the FPN+PAN structure for multi-scale feature fusion, but ignores the spatial details of low-level features, resulting in insufficient ability to extract building edge information and small object instances.

[0041] 2) The lengthy feature transmission path leads to information degradation and high computational overhead: When the PAN structure transmits high-level spatial detail information from the lower layer to the lower layer, some detail information is easily lost after multi-level convolution and sampling, and the computational complexity increases.

[0042] 3) Background noise interferes with feature expression: The feature map after fusing low-level information contains a large amount of background information in non-building areas (such as vegetation and roads), which makes the model susceptible to background noise interference during instance segmentation, leading to problems such as false detection or blurred building outlines.

[0043] Because the embodiment of the present application utilizes semi-supervised learning to construct a building extraction model, when using the YOLOv8 network as the pseudo-label extraction model (i.e., the base model) and feature extraction model (i.e., the first initial building extraction model) for semi-supervised learning, the erroneous information generated by the YOLOv8 network in the building target detection and instance segmentation tasks is continuously amplified through the self-reinforcement mechanism of the pseudo-labels, resulting in a decrease in the extraction performance of the entire semi-supervised learning method. Based on this, the first initial building model provided in the embodiment of the present application utilizes a feature enhancement model based on YOLOv8, and this feature enhancement model based on YOLOv8 expands the fusion range to the P1-P5 feature maps, and introduces a multi-scale feature fusion unit and an edge feature unit to reduce the amount of computation.

[0044] Specifically, the first initial building extraction model includes a backbone network module, a neck module and a head module, and the backbone network module is connected to the neck module; the neck module includes an upsampling unit, a multi-scale feature fusion unit and an edge feature unit, and the upsampling unit includes several cascaded upsampling layers, and the upsampling layers of the preset number of layers are all connected to the multi-scale feature fusion unit, and the multi-scale feature fusion unit is connected to the head module; the first upsampling layer of the two adjacent upsampling layers in the preset number of layers is connected to the next upsampling layer through the edge feature unit, and the last upsampling layer is connected to the head module through the edge feature unit, and the upsampling layers of the preset number of layers before the last upsampling layer are connected to the head module. In addition, if Figure 3 As shown, the backbone network module can adopt the backbone network module in the YOLOv8 network, which includes CSB, C2F and SEEP; the head module includes three classification heads.

[0045] It should be noted that the number and preset number of downsampling layers included in the upsampling unit can be set according to actual needs. In the embodiment of the present application, the upsampling unit includes 5 upsampling layers and the preset number is 3 for illustration, that is, the backbone network model extracts 5 feature maps, which are recorded as P1-P5 feature maps in order from large to small image size. The upsampling unit fuses the P1-P5 feature maps, and the multi-scale feature fusion unit is used to fuse the feature maps extracted by the last three upsampling layers (i.e., P1 feature map, P2 feature map, and P3 feature map).

[0046] In the embodiments of this application, Figure 4 As shown in the figure, the upsampling unit fuses all feature maps extracted by the backbone network model, avoiding the problem that the edge, texture, and spatial position information contained in the P1 feature map and the P2 feature map are ignored due to the feature fusion of only the P3-P5 feature maps in the YOLOv8 network. It captures high-resolution details and spatial information, and improves the overall accuracy of target extraction and small target detection capabilities.

[0047] like Figure 5 As shown, the Triple Feature Encoder Model (TFE) includes a first processing branch, a second processing branch, a third processing branch, and a fusion layer. The first processing branch is used to downsample the P1 feature map using an average pooling (AvgPooling) module to make the image size of the downsampled PI feature map equal to the image size of the P2 feature map, helping to maintain high-resolution features and prevent loss of small object information. The downsampled PI feature map is then convolved with the CBS module to enhance the model's nonlinear capabilities and model complexity. The second processing branch is used to perform a 1×1 convolution operation on the P2 feature map to maintain model complexity. The second processing branch is used to upsample the P3 feature map using the nearest neighbor interpolation method to make the image size of the upsampled P3 feature map equal to the image size of the P2 feature map, helping to maintain the richness of local features in the low-resolution image. The upsampled P3 feature map is then subjected to a 1×1 convolution operation to maintain model complexity. The fusion layer is used to splice the feature maps output by the first, second, and third processing branches in the spatial dimension along the channel dimension. Of course, in practical applications, the multi-scale feature fusion unit can also adopt other structures, for example, without performing convolution operations on the first, second, and / or third processing branches.

[0048] The embodiment of the present application deploys a multi-scale feature fusion unit in the neck module, and directly convolves, upsamples or downsamples the P1-P3 feature maps through the multi-scale feature fusion unit and then splices them, thereby avoiding the computational overhead of layer-by-layer upsampling and multiple feature transformations, thereby improving the inference efficiency of the model; retains the high-resolution features of P1 and P2, and downsamples through AvgPooling to avoid the loss of detail information that may be caused by multiple convolution downsampling; directly splices in the spatial dimension, so that low-level detail information and high-level semantic information can be more directly fused, avoiding the loss of long-path information transmission, thereby enhancing the effectiveness of feature transmission.

[0049] Furthermore, since the neck module integrates the P1 feature map and the P2 feature map, although the P1 feature map and the P2 feature map can bring more detailed information, the P1 feature map and the P2 feature map will also contain a large amount of background information, which will reduce the focusing ability of the model and reduce its attention to the target area. The model will mistakenly focus on non-target areas, affecting the detection accuracy. For this reason, the embodiment of the present application sets an edge feature unit in the neck module, and extracts the significant targets in the feature map through the edge feature unit, so that the model pays more attention to the target area and separates the target area from the background area. Among them, the edge feature unit can use an edge operator to perform feature edge detection, and the edge operator can use a Prewitt operator, a Laplacian operator, a Sobel operator, etc.

[0050] In the embodiment of the present application, the Prewitt operator is selected as the edge operator used by the edge feature unit. The Prewitt operator is an edge detection method based on first-order differentials. It uses the grayscale difference between the upper and lower and left and right neighboring points of a pixel to reach an extreme value at the edge to detect the edge, thereby removing some pseudo-edges. The Prewitt operator uses convolution operations to calculate the gradient of the image in the x direction (horizontal) and the y direction (vertical), and determines the edge position by the gradient size. The calculation steps of the Prewitt operator can be: 1) Set the x-direction convolution kernel ( ): = .

[0051] 2) Set the y-direction convolution kernel ( ) = .

[0052] 3) Use the x-direction convolution kernel and the y-direction convolution kernel to calculate the horizontal gradient and vertical gradient , where the horizontal gradient and vertical gradient The calculation formulas are: , , in, represents the horizontal gradient, represents the vertical gradient, represents the convolution kernel in the x direction, represents the convolution kernel in the y direction, Representing an image Pixels in .

[0053] 4) Using horizontal gradient and vertical gradient Calculate the gradient of each pixel, where the gradient of the pixel is calculated as: , in, Represents the gradient of a pixel.

[0054] Based on this, the edge feature unit (denoted as Prewitt) in the embodiment of the present application combines the Prewitt operator, weight learning and attention weighting to enhance the edge features of the building in the head module. Specifically, Figure 6 As shown, the edge feature unit includes an edge feature extraction block, a splicing block, a channel attention block, a normalization block, a convolution block and a fusion block. The edge feature extraction block, the splicing block, the channel attention block, the normalization block and the fusion block are cascaded in sequence. The convolution block is connected to the fusion block. The input items of the convolution block and the edge feature extraction block are both input items of the edge feature unit. The edge feature extraction block is used to perform a convolution operation on the input item using the Prewitt operator in the spatial dimension to extract edge features in the input item.

[0055] The edge feature map extraction module is used to perform convolution operations on the feature map using the Prewitt operator in the spatial dimension to extract the edge features in the feature map. , Represents the channel dimension. An additional learnable parameter is also set (The initial value is 0.001), which is used to dynamically adjust the edge features in each channel during training The weights of , enable the network to adaptively learn the importance of edge information.

[0056] The splicing block is used to splice edge features of different channels in the channel dimension to obtain rich edge representation.

[0057] The channel attention block is used to adaptively assign weights to different channels and perform weighting in the channel dimension based on the channel weights. This allows the model to focus on more useful edge features and suppress redundant or irrelevant information. On the other hand, it can enhance significant edge information and reduce noise interference, thereby highlighting key edge information while reducing background noise.

[0058] The normalization block uses the Sigmoid function for nonlinear normalization, and normalizes the edge weight to The range makes the edge weight smoother and avoids sudden changes. At the same time, it ensures that the weight value is within a reasonable range and improves the stability of the model.

[0059] The convolution block can use the ConvBNSiLU module to increase the nonlinear ability of the model, making the network more expressive during the learning process and avoiding gradient disappearance or overfitting during network training.

[0060] The fusion block is used to weight the feature map using edge weights. Specifically, the output of the normalization block is applied to the output of the convolution block, so that areas with high edge weight values ​​(such as the target area) are enhanced and areas with low edge weight values ​​(such as the background area) are suppressed. The fusion process of the fusion block can be expressed as: , in, represents the output item of the normalization block (i.e., the edge weight value), represents the output item of the convolution block (i.e., the feature map after convolution), Represents the output item of the edge feature unit.

[0061] The edge feature unit in the embodiment of the present application uses an edge feature weighting method similar to the attention mechanism to enhance the edge features of the building, allowing the model to learn to focus on the prominent areas of the target, reduce background interference, and improve the building detection accuracy and optimize the instance segmentation effect while ensuring lightweight.

[0062] It should be noted that, in practical applications, the first initial building extraction model can also adopt other deep learning models according to actual needs. For example, if timeliness is the primary requirement, the YOLOv8 network can be directly used as the first initial building extraction model.

[0063] S20 , performing weak enhancement and strong enhancement on the unlabeled training data in the unlabeled training dataset respectively to obtain first training data and second training data corresponding to the unlabeled training data.

[0064] Specifically, weak augmentation and strong augmentation are two different augmentation methods. That is, two different augmentations are applied to the unlabeled training data to obtain first training data and second training data. The first and second training data are perturbed, allowing the building extraction model to learn a stable representation that is independent of minor data perturbations, while reducing its reliance on specific local details, thereby improving the model's robustness and generalization performance.

[0065] Weak enhancement is used to maintain the basic structure of the image, while strong enhancement is used to expand the potential feature space of the sample. By optimizing and reducing the loss between weak and strong enhancement, the model's adaptability and robustness to complex data distributions are enhanced. Weak enhancement can include geometric deformation and / or color perturbation, while strong enhancement can include geometric transformation, color transformation, and / or partial area occlusion.

[0066] In one embodiment, the weak enhancement method may include geometric deformation and color perturbation, which preserves the structural information of the image with relatively few transformations. For example, the weak enhancement method is: Random horizontal or vertical flip: Flip the image horizontally or vertically with a 50% probability.

[0067] Random horizontal or vertical movement: Move horizontally or vertically with a probability of up to 12.5%.

[0068] Color transformation: Perform HSV image space conversion on the image with a probability of 50%.

[0069] In one embodiment, weak augmentation uses a combination of RandAugment and Cutout. Cutout enhances the model's generalization capabilities by randomly occluding regions of the image (filling them with zeros or the mean). The core idea is to improve model robustness by forcing the model to infer target features from the remaining context by randomly occluding local regions of the image. Furthermore, by constraining the features of different occluded versions, the model's reliance on specific local details is reduced.

[0070] RandAugment uses two hyperparameters M and N to control the magnitude of image transformation. M represents the transformation strength, which is used to control the magnitude of each data augmentation operation. N represents the number of transformations, which is used to control the number of augmentation transformations applied to each image. RandAugment reduces the cost of manual adjustment by automatically searching for augmentation strategies, while striking a balance between enhancing diversity and avoiding overfitting. For example, the transformation strategies of RandAugment transformation are shown in Table 1, and the enhanced images based on each transformation strategy of RandAugment transformation and the enhanced images based on Cutout are shown in Table 1. Figure 7shown.

[0071] Table 1 Transformation strategy of RandAugment transformation

[0072] S30, extracting a first prediction profile, a classification confidence, and a mask prediction score of the first training data using the basic model, and screening the first prediction profile based on the classification confidence and the mask prediction score to obtain a pseudo label for the first training data; Specifically, the first training data is weakly augmented training data, which is used to determine the pseudo labels corresponding to the unlabeled data corresponding to the first training data. In other words, the base model is used as a teacher model, and the first training data (i.e., the weakly augmented unlabeled training data) is used to provide pseudo labels for the unlabeled training data.

[0073] The classification confidence is used to indicate the credibility of the prediction box predicted by the base model and is used as a noise indicator. The mask prediction score is used to reflect the credibility of the mask predicted by the base model and is used as a spatial consistency metric. The classification confidence and mask prediction scores are used as the basis for filtering pseudo-labels. A dual-threshold strategy is constructed based on the classification confidence and mask prediction scores, and this dual-threshold strategy is used to filter the first prediction contour predicted by the base model to improve the reliability of the filtered pseudo-labels.

[0074] In one embodiment, the screening of the first predicted profile based on the classification confidence and the mask prediction score to obtain a pseudo label for the first training data specifically includes: comparing the classification confidence of the predicted contour in the first predicted contour with a confidence threshold, and simultaneously comparing the mask prediction score of the predicted contour with a mask score threshold; If the classification confidence is greater than the confidence threshold and the mask prediction score is greater than the mask score threshold, the predicted contour is retained to obtain a pseudo label for the first training data.

[0075] Specifically, if Figure 8 As shown, the confidence threshold and the mask score threshold are both pre-set. The confidence threshold is used as the screening basis for classification confidence screening, and the mask score threshold is used as the screening basis for mask prediction score screening. When the classification confidence is greater than the confidence threshold, it means that the prediction box of the first predicted contour meets the requirements. When the mask prediction score is greater than the mask score threshold, it means that the mask corresponding to the first predicted contour meets the requirements, and the first predicted contour is considered to be a reliable prediction result. Based on this, the pseudo label can be expressed as: , in, represents a pseudo label, represents the first predicted contour, represents the classification confidence, represents the mask score threshold, represents the confidence threshold, Represents the mask score threshold.

[0076] The embodiment of the present application adopts a dual-threshold strategy for pseudo-label screening, suppresses the "false positive" prediction value of positioning deviation through classification confidence, and suppresses the "missed detection" target corresponding to low-quality mask through mask prediction score, thereby improving the reliability of pseudo-labels.

[0077] S40, extracting a second predicted outline of the second training data using a second initial building extraction model; Specifically, the second initial building extraction model is obtained by adding a feature perturbation module to the first initial building extraction model. The second initial building extraction model is used to extract the second predicted contour corresponding to the second training data. In other words, the second initial building extraction model serves as a student model, and the pseudo labels generated by the base model (i.e., the teacher model) serve as the supervision signal for the second initial building extraction model. The knowledge learned by the base model is transferred to the second initial building extraction model, thereby achieving training for the second initial building extraction model.

[0078] The second initial building extraction model is obtained by adding a feature perturbation module to the first initial building extraction model. The feature perturbation module is used to add perturbations to the feature maps extracted by the first initial building extraction model to achieve synchronous perturbations of the training data (i.e., input perturbations) and the feature maps corresponding to the training data (i.e., feature perturbations). This can further enhance the building extraction model's ability to extract global semantic features and improve generalization.

[0079] The feature perturbation module is used to insert a channel dropout operation in the feature space. This operation maintains consistent predictions at the image and feature levels of unlabeled training data, allowing the input perturbation and feature perturbation to complement each other. However, in practical applications, because channel dropout only operates on the channel dimension, it randomly discards some feature channels without explicitly destroying the spatial continuity of the feature map. In convolutional neural networks, features typically have strong spatial correlations, so the model can still rely on local regions for prediction, which limits its ability to model global features.

[0080] Based on this, in one embodiment, the feature perturbation module uses DropBlock, a feature-space-based regularization method, to further enhance the effectiveness of feature perturbation. DropBlock is a regularization method designed to enhance model robustness by structuredly discarding continuous regions in feature maps. Its core idea is to force the network to learn a more global feature representation by disrupting the spatial continuity of feature maps.

[0081] like Figure 9 As shown in the figure, the working process of DropBlock is to preset the size of the occluded area, then randomly select the center point on the feature map of the image, and generate the occlusion area based on the size of the occluded area (such as filling it with 0) to generate a mask map. Finally, the mask map is multiplied by the feature map to obtain the perturbed feature map. The perturbed feature map discards the feature information of the occluded area. The calculation process of DropBlock can be: 1) Calculate the central point sampling probability γ, where the calculation formula for γ can be: , in, represents the feature retention probability (similar to the retention probability of Dropout), and Represents the height and width of the feature map, Indicates the size of the occluded area.

[0082] 2) Generate a binary mask where each position is independently activated with probability (as the center of the dropped block), where the binary mask is represented as: , in, Represents a Bernoulli distribution with probability Generate 1 with probability Generate 0, Represents the mask value, Indicates that the position is selected as the center of the discarded block, Indicates that the location was not selected as the center of the discarded block.

[0083] 3) Expand the center point to The center mask block of The center mask block of is represented as: , in, represents the center mask block, represents a binary mask, Indicates size The convolution kernel is all 1, Represents a convolution operation that requires dilation to maintain size.

[0084] 4) Invert the block mask to obtain the mask map of the retained area. The mask map is expressed as: , in, Represents a mask image.

[0085] S50: Train the second initial building extraction model according to the second predicted outline and the pseudo label to obtain a building extraction model.

[0086] Specifically, after obtaining the second predicted contour and pseudo-label, a loss function can be constructed based on the second predicted contour and pseudo-label, and the second initial building extraction model can be trained using this loss function. However, because the first training data and the second training data were obtained using different data augmentation techniques (weak augmentation for the first training data and strong correctness for the second training data), the pseudo-label and the second predicted contour may be spatially inconsistent. Therefore, before constructing the loss function based on the second predicted contour and pseudo-label, a spatial alignment mechanism can be used to spatially align the pseudo-label with the second predicted contour to ensure a mapping relationship between the pseudo-label and the second predicted contour.

[0087] In one embodiment, the alignment process of the pseudo label and the second predicted contour may be: First, obtain the scaling factor between the weak enhancement corresponding to the first training data and the strong enhancement corresponding to the second training data , rotation angle , flip parameters and translation offset , and according to the scaling factor , rotation angle , flip parameters and translation offset Calculate the global affine transformation matrix , where the global affine transformation matrix is The calculation formula can be: , in, and Indicates the flip parameter.

[0088] Secondly, the pseudo-label is mapped to the input space of the second initial building extraction model using the global affine transformation matrix to align the pseudo-label with the second predicted contour, wherein the mapped pseudo-label is: , , in, represents the predicted box in the pseudo label, represents the mask map in the pseudo label, represents the predicted box in the mapped pseudo-label, The mask map in the mapped pseudo-label.

[0089] Finally, after the second initial building extraction model is trained using the unlabeled training data set, the feature perturbation in the trained second initial building extraction model is removed to obtain a building extraction model, that is, the building extraction model does not include the feature perturbation.

[0090] S60: Acquire a building satellite image, and use the building extraction model to extract the building outline of the building in the building satellite image.

[0091] Specifically, the building satellite image can be acquired from the domestically produced Gaofen-7 satellite, and the building extraction model is trained through steps S10-S50. The building satellite image serves as input to the building extraction model, which outputs the building outlines corresponding to the satellite image. Furthermore, after acquiring the building outlines, the outline box (i.e., boundary) of each building instance within the building outline can be extracted and preprocessed to further improve the accuracy of building instance segmentation. Preprocessing can include regularization and / or vectorization.

[0092] In one embodiment, after acquiring the building satellite image and extracting the building outlines of the buildings in the building satellite image using the building extraction model, the method further includes: Extracting connected area boundary lines in the building outline to form a closed boundary point sequence; Key points representing structural turning points are selected from the boundary point sequence, and polygonal contours generated from the selected key points are regularized.

[0093] Specifically, the boundary point sequence is a set of boundary points obtained by extracting connected areas from the building outline. That is, when extracting connected area boundary lines from the building outline, several contour point sequences will be extracted from the building outline, and each contour point sequence is a boundary point sequence of a connected area.

[0094] In one embodiment, the connected region boundary line is extracted from the binary image corresponding to the building outline using the eight-neighborhood contour tracking algorithm. The specific extraction process includes first finding the first foreground pixel (value 255) in the binary image and setting it as As a starting point, then from Start and search for the next boundary point in its field in a fixed direction (counterclockwise) , and Continue searching for its adjacent points as the center and construct boundary sequences in sequence , until it returns to the starting point and closes, so as to obtain several ordered point sets representing the boundaries of the building, that is, several closed boundary point sequences.

[0095] Furthermore, after obtaining the boundary point sequence, key points can be selected from the boundary point sequence and the boundary contour can be regenerated based on the key points to improve contour expression efficiency and structural clarity. Key points include boundary points used to represent structural transitions, boundary points used for structural transitions, and partial boundary points between two connected boundary points used for structural transitions.

[0096] In one embodiment, the Douglas-Peucker algorithm can be used to simplify the building to obtain key points representing structural transitions. The goal of the Douglas-Peucker algorithm is to remove redundant points, retain key points representing structural transitions, and generate a polyline (polygon) that approximates the original outline while ensuring that the overall shape error of the outline does not exceed a given threshold. Specifically, let the boundary point sequence corresponding to the boundary line of a connected pipe region be , the process of selecting key points by Douglas-Peucker algorithm is: 1) Take the first boundary point and the last boundary point As the initial line segment; 2) Calculate the boundary point sequence and The vertical distance from the middle boundary point to the initial line segment; 3) Find the boundary point with the largest vertical distance , record the vertical distance ; 4) If (preset error threshold), then retain the boundary point , and divide the initial line segment into two segments and , and then respectively and As the initial line segment, repeat step 2); 5) If , indicating that the intermediate boundary points have little influence on the overall shape, all intermediate boundary points are discarded to complete the key point selection.

[0097] After completing the key point selection, a polyline outline will be constructed based on the selected key points to obtain the polygonal outline corresponding to the building outline. The number of boundary points of the polygonal outline is less than the number of boundary points of the building outline, but the geometric shape of the polygonal outline remains consistent with the geometric shape of the building outline.

[0098] Furthermore, in remote sensing images, especially high-resolution images, building outlines often have obvious directional characteristics, such as common orthogonal buildings (0° / 90°) and sloping roof buildings (45° / 135°). However, the building outlines output by deep learning models are pixel-level raster edges. After being converted into polygons, the direction of the edge segments may have the following problems: slight angle offsets (such as 87°, 91°, etc.); unstructured stray directions (such as 78.6°, 112.3°, etc.); and inconsistent multi-directional combinations. To this end, when regularizing the polygonal outlines generated from key points, the direction angles of each segment in multiple polygonal outlines can be aligned to the main direction and regularized to achieve the purpose of unifying the building edge direction and improving the regularity of the outline.

[0099] Exemplarily, the step of regularizing the polygonal contour generated from the selected key points specifically includes: Draw contour segments based on the obtained key points and calculate the direction angle of each contour segment; quantizing the direction angles using a preset main direction sequence to merge the direction angles into the nearest main direction; The contour line segments are geometrically reconstructed according to the quantized direction angles, and a polygonal contour is generated based on the reconstructed contour lines.

[0100] Specifically, the direction angle of the contour line segment refers to the angle between the contour line segment and the positive direction of the horizontal axis. The calculation formula of the direction angle of the contour line segment can be: , in, Represents the direction angle of the contour segment, ( , ) represents an endpoint of a contour segment, ( , ) represents the other endpoint of the contour segment.

[0101] The preset main direction sequence is pre-built according to the structural characteristics of the building, which includes several main direction angles. For example, for an orthogonal building, the preset main direction sequence is For sloped roofs or complex structures, preset main direction sequence .

[0102] Quantizing the direction angle using a preset main direction sequence means selecting the main direction closest to the direction angle of the contour segment in the preset main direction sequence. The closest main direction can be the main direction with the smallest angular difference from the direction angle of the contour segment. The closest main direction can be expressed as: ,in, Indicates the direction angle of the contour segment The closest main direction of Indicates the preset main direction sequence, Indicates the main direction in the preset main direction sequence. Then, the angle difference between the direction angle of the closest main direction and the contour segment is calculated. Deviation from preset threshold (like ~ ) for comparison, if , then the direction angle of the contour segment is quantified as , , the direction angle of the contour segment remains unchanged.

[0103] Furthermore, after quantizing the direction angle of each contour segment, the direction angle of the contour segment quantized to the closest main direction is reconstructed based on the quantized direction angle, wherein the reconstruction process can adopt the central axis rotation method, retain the midpoint of the segment, and rotate the original segment as a whole to align its direction to the quantized direction.

[0104] In summary, this embodiment provides a method for extracting and regularizing building outlines from satellite images. The method includes constructing a base model using a first initial building extraction model based on YOLOv8 and a labeled training dataset, using weak enhancement and the base model to construct pseudo labels for unlabeled training data using a double threshold method, extracting a second predicted outline using a second building extraction model that adds feature perturbations to the first initial building extraction model, training the building extraction model based on the pseudo labels and the second predicted outline, and extracting building outlines using the building extraction model. This method improves the accuracy of GF-7 building extraction under labeled data conditions and achieves efficient and low-cost GF-7 image building extraction. Furthermore, the first initial building extraction model based on YOLOv8 introduces scale feature fusion optimization and edge features for feature enhancement, effectively improving the accuracy of instance segmentation tasks.

[0105] Based on the above-mentioned method for extracting and regularizing building outlines based on satellite images, this embodiment provides a device for extracting and regularizing building outlines based on satellite images, such as Figure 10 As shown, the building outline extraction and regularization device based on satellite images specifically includes: A pre-training module 100 is used to train a first initial building extraction model using a labeled training data set to obtain a basic model; An enhancement module 200 is configured to perform weak enhancement and strong enhancement on the unlabeled training data in the unlabeled training dataset, respectively, to obtain first training data and second training data corresponding to the unlabeled training data; a pseudo-label extraction module 300 for extracting a first prediction profile, a classification confidence, and a mask prediction score of the first training data using the basic model, and filtering the first prediction profile based on the classification confidence and the mask prediction score to obtain a pseudo-label for the first training data; A predicted contour extraction module 400 is configured to extract a second predicted contour of the second training data using a second initial building extraction model, wherein the second initial building extraction model is obtained by adding a feature perturbation module to the first initial building extraction model; A training module 500 is configured to train the second initial building extraction model according to the second predicted contour and the pseudo label to obtain a building extraction model; The building outline extraction module 600 is used to obtain a building satellite image and extract the building outlines of the buildings in the building satellite image using the building extraction model.

[0106] Based on the above-mentioned method for extracting and regularizing building outlines based on satellite images, this embodiment provides a computer-readable storage medium, which stores one or more programs. The one or more programs can be executed by one or more processors to implement the steps in the method for extracting and regularizing building outlines based on satellite images as described in the above-mentioned embodiment.

[0107] Based on the above-mentioned building outline extraction and regularization method based on satellite images, this application also provides a terminal device, such as Figure 11 As shown, it includes at least one processor 20; a display screen 21; and a memory 22. It may also include a communications interface 23 and a bus 24. The processor 20, display screen 21, memory 22, and communications interface 23 can communicate with each other via bus 24. The display screen 21 is configured to display a preset user guidance interface in the initial setup mode. The communications interface 23 can transmit information. The processor 20 can invoke logic instructions in the memory 22 to execute the method described in the above embodiment.

[0108] In addition, the logic instructions in the memory 22 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product.

[0109] The memory 22, as a computer-readable storage medium, can be configured to store software programs or computer-executable programs, such as program instructions or modules corresponding to the methods in the embodiments of the present disclosure. The processor 20 executes the software programs, instructions, or modules stored in the memory 22 to perform functional applications and data processing, thereby implementing the methods in the above embodiments.

[0110] The memory 22 may include a program storage area and a data storage area. The program storage area may store an operating system and at least one application required for a function; the data storage area may store data created based on the use of the terminal device. In addition, the memory 22 may include high-speed random access memory and non-volatile memory. For example, various media that can store program code, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, may also be transient storage media.

[0111] In addition, the specific process of loading and executing the multiple instructions in the storage medium and the processor in the terminal device has been described in detail in the above method and will not be described here one by one.

[0112] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for extracting and regularizing building outlines based on satellite images, characterized in that: The satellite image-based building outline extraction and regularization method specifically includes: Training the first initial building extraction model using the labeled training data set to obtain a base model; Performing weak enhancement and strong enhancement on the unlabeled training data in the unlabeled training dataset respectively to obtain first training data and second training data corresponding to the unlabeled training data; Extracting a first prediction profile, a classification confidence, and a mask prediction score of the first training data using the base model, and filtering the first prediction profile based on the classification confidence and the mask prediction score to obtain a pseudo label for the first training data; Extracting a second predicted profile of the second training data using a second initial building extraction model, wherein the second initial building extraction model is obtained by adding a feature perturbation module to the first initial building extraction model; Training the second initial building extraction model according to the second predicted contour and the pseudo label to obtain a building extraction model; A building satellite image is acquired, and building outlines of buildings in the building satellite image are extracted using the building extraction model.

2. The method for extracting and regularizing building outlines based on satellite images according to claim 1, wherein: The first initial building extraction model includes a backbone network module, a neck module and a head module, and the backbone network module is connected to the neck module; the neck module includes an upsampling unit, a multi-scale feature fusion unit and an edge feature unit, the upsampling unit includes several cascaded upsampling layers, and the upsampling layers of the subsequent preset number of layers are all connected to the multi-scale feature fusion unit, and the multi-scale feature fusion unit is connected to the head module; the previous upsampling layer of the two adjacent upsampling layers in the subsequent preset number of layers is connected to the next upsampling layer through the edge feature unit, and the last upsampling layer is connected to the head module through the edge feature unit, and the upsampling layers of the previous preset number of layers of the final upsampling layer are connected to the head module.

3. The method for extracting and regularizing building outlines based on satellite images according to claim 2, wherein: The edge feature unit includes an edge feature extraction block, a splicing block, a channel attention block, a normalization block, a convolution block and a fusion block. The edge feature extraction block, the splicing block, the channel attention block, the normalization block and the fusion block are cascaded in sequence, and the convolution block is connected to the fusion block. The input items of the convolution block and the edge feature extraction block are both input items of the edge feature unit. The edge feature extraction block is used to perform a convolution operation on the input item using a Prewitt operator in the spatial dimension to extract edge features in the input item.

4. The method for extracting and regularizing building outlines based on satellite images according to claim 1, wherein: The step of screening the first predicted profile based on the classification confidence and the mask prediction score to obtain a pseudo label for the first training data specifically includes: comparing the classification confidence of the predicted contour in the first predicted contour with a confidence threshold, and simultaneously comparing the mask prediction score of the predicted contour with a mask score threshold; If the classification confidence is greater than the confidence threshold and the mask prediction score is greater than the mask score threshold, the predicted contour is retained to obtain a pseudo label for the first training data.

5. The method for extracting and regularizing building outlines based on satellite images according to claim 1, wherein: The weak enhancement includes geometric deformation and / or color disturbance, and the strong enhancement includes geometric transformation, color transformation and / or partial area occlusion.

6. The method for extracting and regularizing building outlines based on satellite images according to claim 1, wherein: After acquiring the building satellite image and extracting the building outlines of the buildings in the building satellite image using the building extraction model, the method further includes: Extracting connected area boundary lines in the building outline to form a closed boundary point sequence; Key points representing structural turning points are selected from the boundary point sequence, and polygonal contours generated from the selected key points are regularized.

7. The method for extracting and regularizing building outlines based on satellite images according to claim 6, wherein: The regularization of generating polygonal contours from the selected key points specifically includes: Draw contour segments based on the obtained key points and calculate the direction angle of each contour segment; quantizing the direction angles using a preset main direction sequence to merge the direction angles into the nearest main direction; The contour line segments are geometrically reconstructed according to the quantized direction angles, and a polygonal contour is generated based on the reconstructed contour lines.

8. A device for extracting and regularizing building outlines based on satellite images, characterized in that: The satellite image-based building outline extraction and regularization device specifically includes: A pre-training module, configured to train a first initial building extraction model using a labeled training dataset to obtain a basic model; An enhancement module is used to perform weak enhancement and strong enhancement on the unlabeled training data in the unlabeled training data set, respectively, to obtain first training data and second training data corresponding to the unlabeled training data; a pseudo-label extraction module, configured to extract a first prediction profile, a classification confidence, and a mask prediction score of the first training data using the base model, and filter the first prediction profile based on the classification confidence and the mask prediction score to obtain a pseudo-label for the first training data; a predicted contour extraction module, configured to extract a second predicted contour of the second training data using a second initial building extraction model, wherein the second initial building extraction model is obtained by adding a feature perturbation module to the first initial building extraction model; a training module, configured to train the second initial building extraction model according to the second predicted contour and the pseudo label to obtain a building extraction model; The building outline extraction module is used to obtain a building satellite image and extract the building outline of the building in the building satellite image using the building extraction model.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps in the satellite image-based building outline extraction and regularization method as described in any one of claims 1 to 7.

10. A terminal device, characterized in that: include: processor and memory; The memory stores a computer-readable program executable by the processor; When the processor executes the computer-readable program, the steps of the method for extracting and regularizing building outlines based on satellite images are implemented.

Images