Building extraction method based on high-resolution network

Abstract

The invention discloses a building extraction method based on a high-resolution network, and belongs to the field of image processing and machine vision detection. In order to solve the problems of void phenomenon, poor boundary precision, missing extraction and mistaken extraction of small-sized buildings and the like existing in high-resolution remote sensing image building extraction of an existing deep learning method, the method adopts a multi-stage network optimization strategy: firstly, obtaining a high-resolution building data set; and expanding the training set by adopting a data enhancement technology of sample random rotation transformation, Gaussian filtering boundary enhancement, HSV-based color space transformation and standard deviation-added random Gaussian noise. Sequentially integrating object context representation (OCR) modules on the HRNet backbone network to reduce internal holes; introducing a boundary refinement post-processing Segfix module to carry out direction-guided pixel-level correction on an output boundary so as to optimize the building boundary; a rotary variable-size window attention (RVSA) mechanism is embedded in a down-sampling layer, and the capacity of capturing small target features is enhanced through a dynamically changing window. And finally, end-to-end training is carried out on the constructed model by using an enhanced data set, accurate and complete extraction of the building is realized, and an efficient and reliable solution is provided for scene applications such as urban planning and disaster assessment.

Description

Building Extraction Method Based on High-Resolution Networks Technical Field

[0001] This invention belongs to the field of image processing and remote sensing information extraction technology, specifically relating to a building extraction method based on high-resolution networks. Background Technology

[0002] With the acceleration of urbanization, the number of buildings has increased dramatically, and their spatial distribution and morphological changes have become increasingly complex. Therefore, efficient and accurate extraction of building information has significant application value in urban planning, disaster assessment, and smart city construction. With the rapid development of remote sensing technology, high-resolution remote sensing imagery has significant advantages in spectral resolution, texture features, and spatial detail. It can clearly present the geometric structure, outline, and surface texture of ground features, providing high-precision data support for ground feature identification, fine classification, and dynamic information updates. It plays an irreplaceable role in the automated extraction and refined interpretation of buildings.

[0003] In recent years, deep learning-based remote sensing image interpretation methods have provided efficient and accurate solutions for building patch information extraction due to their powerful feature learning capabilities, significantly improving the accuracy and robustness of building extraction. Fully convolutional neural networks (FCNs) and their variants (such as U-Net, SegNet, and HRNet) have become mainstream methods. Among them, HRNet, with its multi-resolution branch parallel structure, effectively preserves high-resolution features and demonstrates advantages in building extraction. However, existing deep learning-based methods still suffer from the following key problems: First, the downsampling stage easily leads to feature value loss, resulting in gaps in the extracted building results; second, the lack of effective post-processing mechanisms results in poor accuracy in building boundary extraction, with issues such as blurred boundaries and noise interference; third, insufficient feature learning for small buildings easily leads to missed or incorrect extractions. These problems seriously affect the accuracy and completeness of building extraction, limiting the application of related technologies in real-world scenarios.

[0004] Therefore, there is an urgent need for a method that can solve the above problems and achieve accurate and efficient extraction of buildings from high-resolution remote sensing images. Summary of the Invention

[0005] To address the issues of voids, poor boundary accuracy, and omissions or errors in the extraction of buildings from high-resolution remote sensing images using existing deep learning methods, this invention provides a building extraction method based on a high-resolution network. Based on the high-resolution network HRNet, through multi-stage network model improvement and optimization, it achieves accurate and complete extraction of buildings.

[0006] The high-resolution network-based building extraction method of the present invention includes the following steps: selecting a high-resolution remote sensing image building dataset and expanding the dataset using data augmentation techniques; constructing an improved HRNet detection model, wherein the improved HRNet detection model integrates an object context representation (OCR) module, a boundary refinement post-processing (Segfix) module, and a rotationally variable size attention (RVSA) mechanism; training the improved HRNet detection model end-to-end using the augmented dataset; inputting the high-resolution remote sensing image to be extracted into the trained model, outputting a probability map of the building region, and thresholding the probability map to obtain the final building extraction result.

[0007] Furthermore, data augmentation techniques include random sample rotation transformation, Gaussian filter boundary enhancement, HSV color space-based transformation, and adding random Gaussian noise with standard deviation.

[0008] Furthermore, an OCR module is integrated into the output of HRNet to form an OCR-HRNet model. First, the output feature maps of each layer of the HRNet backbone network are bilinearly interpolated to restore the original resolution. Then, the feature maps of each layer are superimposed. The superimposed cumulative feature map is input into the OCR algorithm to calculate the correlation between pixel features and object region features in the cumulative feature map. The features are weighted and aggregated to generate an enhanced contextual semantic information feature map. Finally, the enhanced feature map is output after 1×1 convolution dimensionality reduction.

[0009] Furthermore, a Segfix post-processing module is added to the output layer of OCR-HRNet to form the OS-HRNet model. First, the Segfix mechanism generates a boundary feature map through the edge prediction branch. It calculates the minimum Euclidean distance from each pixel category to other pixels and determines the boundary pixels using a thresholding method. Then, it generates a direction feature map through the direction prediction branch, predicting the direction of each pixel at all locations to its nearest pixel of the same category. Next, the feature map obtained by multiplying the corresponding pixel matrices of the direction feature map and the boundary feature map is processed by the coordinate offset branch to generate an offset feature map. Finally, the boundary pixels are assigned direction vectors based on the offset feature map, and the unreliable boundary pixels in the original segmentation result are corrected for category and refined for position using a thresholding method to optimize the building boundaries.

[0010] Furthermore, a Rotationally Variable Size Window Attention (RVSA) mechanism is added to the second network of the pre-trained encoder downsampling process of the OS-HRNet model to form the ROS-HRNet network model. First, through a global average pooling layer and a learnable linear transformation, the rotation angle, size scaling factor, and center position offset of each attention window are dynamically predicted to generate overlapping windows with different angles, sizes, and positions. Then, multiple attention calculations are performed on the input features from multiple perspectives to reduce feature loss during the downsampling process and effectively capture the detailed features of small targets such as small buildings.

[0011] Furthermore, the augmented dataset is divided into training, validation, and test sets to train the constructed ROS-HRNet model using the Adam optimizer, a cosine decay learning rate, and a binary cross-entropy loss function. The training process includes forward propagation, loss calculation, and backpropagation until the model converges.

[0012] Based on the above technical solution, the embodiments of the present invention can produce at least the following technical effects:

[0013] (1) The correlation between pixels and object regions was enhanced by the OCR algorithm, realizing the transformation of semantic segmentation from single pixel classification to region feature aggregation, effectively solving the hole problem in the building extraction results and improving the completeness of building extraction.

[0014] (2) The Segfix post-processing mechanism is introduced to perform pixel-level correction of the boundary with direction guidance, which improves the problems of boundary blurring and noise interference, making the boundary of the extraction result more regular and smooth.

[0015] (3) The RVSA attention mechanism is introduced, which enables the model to adaptively focus on features of different scales, directions and locations. In particular, it enhances the ability to capture and retain features of small buildings and reduces the false detection rate and false detection rate of small targets. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0017] Figure 1 is a flowchart of the technology of this invention;

[0018] Figure 2 is a structural diagram of the OCR-HRNet model of the present invention;

[0019] Figure 3 is a flowchart of the OS-HRNet model of the present invention;

[0020] Figure 4 is a structural diagram of the RVSA fusion structure of the present invention; Detailed Implementation

[0021] The technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. In addition, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0022] This invention provides a building extraction method based on high-resolution networks, as shown in Figure 1. The method specifically includes the following steps:

[0023] Step 1: Dataset Creation and Augmentation

[0024] The publicly available Wuhan University Remote Sensing Building Database (WHU) and Massachusetts Building Dataset (Massa) were selected as experimental datasets. The Massa dataset was cropped to 512×512 pixels using a sliding window to match the WHU dataset, and then label binarization was performed.

[0025] Data augmentation techniques are used to expand datasets, including the following methods:

[0026] (1) Random rotation transformation of samples: The image is randomly rotated within the angle range of [0, 180°) to increase the model's ability to identify different building orientations.

[0027] (2) Gaussian filtering edge enhancement: Gaussian filtering is applied to the image to optimize the contrast and sharpness of building edges and enhance boundary features.

[0028] (3) HSV color space transformation: Convert the image from RGB color space to HSV color space, and use its advantages in distinguishing hue, saturation and brightness to enhance the spectral differences between buildings and other background objects.

[0029] (4) Random Gaussian noise addition: Random Gaussian noise with a standard deviation of 0.25 is added to the image to simulate real imaging conditions such as sensor noise, thereby improving the robustness and generalization performance of the model.

[0030] Step 2: Construct the OCR-HRNet model

[0031] As shown in Figure 2, intermediate feature maps are extracted using HRNet and upsampled using bilinear interpolation to restore them to the same resolution as the input image, serving as pixel features. Another set of intermediate feature maps is extracted from HRNet, and after upsampling and 1×1 convolution, coarse segmentation feature maps are obtained. Based on the class probabilities of pixels in the coarse segmentation feature maps, the pixel feature representations are weighted to obtain object region features for each class. The correlation weight coefficients between pixel features and object region features are calculated, and the contextual semantic information features are obtained by weighted summation. The contextual semantic information feature representations and pixel feature representations are weighted and aggregated along the channel dimension, and then subjected to 1×1 convolution to obtain the enhanced feature representation, i.e., the optimized feature map.

[0032] Step 3: Construct the OS-HRNet model

[0033] As shown in Figure 3, a Segfix post-processing module is added to the output layer of OCR-HRNet. First, the Segfix edge prediction branch processes the feature map output by the model through a network structure combination of 1×1×256 convolution + batch normalization layer + ReLU activation function (Conv_BN_ReLU) and 1×1×2 convolution and upsampling (Conv+Upsample) to obtain a boundary feature map. The boundary feature map is determined by calculating the minimum Euclidean distance from each pixel category to other pixels and using a thresholding method. Essentially, it is a binary boundary mask map, where 1 represents the boundary and 0 represents other categories. The Segfix orientation prediction branch obtains an orientation feature map through Conv_BN_ReLU(1×1×256)+Conv(1×1×m)+Upsample, predicting the orientation m of the nearest pixel of the same category at all positions. m is quantized into 8 orientations in the range [0, 2). Then, the feature map resulting from the product of the pixel matrices of the orientation feature map and the boundary feature map is processed through a coordinate offset branch to generate an offset feature map Q (H×W×2). Different directions are mapped to different coordinate offsets. For example, the right direction with m=8 can be mapped to (1,1). Coordinate correction is then used to adjust the corresponding pixels in the original prediction result. Finally, based on the offset feature map, orientation vectors are assigned to the boundary pixels. A thresholding method is used to correct the category and refine the position of unreliable boundary pixels in the original segmentation result, thereby optimizing the building boundaries.

[0034] Step 4: Construct the ROS-HRNet model

[0035] In the second branch of the pre-trained encoder downsampling process of the OS-HRNet model, a Rotationally Variable-Size Window Attention (RVSA) module is added. First, through a global average pooling layer and a learnable linear transformation, the rotation angle, size scaling factor, and center position offset of each attention window are dynamically predicted to generate overlapping windows with different angles, sizes, and positions. The specific steps are as follows:

[0036] Given an input feature F, it is divided into several non-overlapping windows, each with a feature F (where s is the window size). F is processed by Global Average Pooling (GAP), and combined with linear layer prediction, the center offset (Δx, Δy), scaling factor (s, s), and rotation angle θ of each window are obtained. The specific formula is: Δx, Δy, s, s, θ) = Linear(GAP(F))

[0037] The corner coordinates of the transformed window are calculated based on the above parameters, and then the key features and value features are obtained to participate in the window attention calculation. Then, by performing multiple attention calculations on the input features from multiple perspectives, the feature loss during the downsampling process is reduced, and the detailed features of small targets such as small buildings are effectively captured.

[0038] Step 5: Model Training

[0039] The augmented dataset was randomly divided into training, validation, and test sets in a 7:2:1 ratio. Based on the constructed ROS-HRNet model, the training hyperparameters were set as follows: 100 iterations, 16 samples, a maximum learning rate of 4e-3, and a minimum learning rate of 4e-3*0.01. The Adam optimizer was used, with a cosine decay strategy for the learning rate and a binary cross-entropy loss function. The model was trained on a Linux system using the PyTorch framework. The training process included forward propagation, loss calculation, and backpropagation until the model converged.

[0040] Step 6: Building Extraction

[0041] After training, the high-resolution remote sensing image to be processed is input into the ROS-HRNet model, which outputs a probability map of the building area. The probability map is then thresholded to obtain the final building extraction result.

[0042] This invention achieves high-precision and robust detection of small target defects through hardware optimization, dataset construction, and algorithm improvement. It can be directly applied to automated quality inspection in industrial production lines and has significant engineering application value.

[0043] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A building extraction method based on high-resolution networks, characterized in that, Includes the following steps: A high-resolution remote sensing imagery dataset of buildings was selected, and data augmentation techniques were used to expand the dataset. An improved HRNet detection model was constructed, which integrates an Object Context Representation (OCR) module, a Segfix boundary refinement post-processing module, and a Rotationally Variable Size Attention (RVSA) mechanism. The improved HRNet detection model was trained end-to-end using the augmented dataset. The high-resolution remote sensing imagery to be extracted was input into the trained model, and the model output a probability map of the building region. The probability map was then thresholded to obtain the final building extraction result.

2. The building extraction method based on high-resolution networks according to claim 1, characterized in that, The data augmentation techniques include random sample rotation transformation, Gaussian filter boundary enhancement, HSV color space-based transformation, and adding random Gaussian noise with standard deviation.

3. The building extraction method based on high-resolution networks according to claim 1, characterized in that, An OCR module is integrated into the output of HRNet to form an OCR-HRNet model. First, the output feature maps of each layer of the HRNet backbone network are bilinearly interpolated to restore the original resolution. Then, the feature maps of each layer are superimposed. The superimposed cumulative feature map is input into the OCR algorithm to calculate the correlation between pixel features and object region features in the cumulative feature map. The features are weighted and aggregated to generate an enhanced contextual semantic information feature map. Finally, the enhanced feature map is output after 1×1 convolution dimensionality reduction.

4. The building extraction method based on high-resolution networks according to claim 1, characterized in that, A Segfix post-processing module is added to the output layer of OCR-HRNet to form the OS-HRNet model. First, the Segfix mechanism generates a boundary feature map through the edge prediction branch. It calculates the minimum Euclidean distance from each pixel category to other pixels and determines the boundary pixels by using a thresholding method. Then, it generates a direction feature map through the direction prediction branch, predicting the direction of each pixel at all positions to its nearest pixel of the same category. Next, the feature map obtained by multiplying the corresponding pixel matrices of the direction feature map and the boundary feature map is processed by the coordinate offset branch to generate an offset feature map. Finally, the boundary pixels are assigned direction vectors based on the offset feature map, and the unreliable boundary pixels in the original segmentation result are corrected for category and refined for position by a thresholding method, thereby optimizing the building boundaries.

5. The building extraction method based on high-resolution networks according to claim 1, characterized in that, In the second network of the pre-trained encoder downsampling process of the OS-HRNet model, a Rotationally Variable Size Window Attention (RVSA) mechanism is added to form the ROS-HRNet network model. First, through a global average pooling layer and a learnable linear transformation, the rotation angle, size scaling factor, and center position offset of each attention window are dynamically predicted to generate overlapping windows with different angles, sizes, and positions. Then, multiple attention calculations are performed on the input features from multiple perspectives to reduce feature loss during the downsampling process and effectively capture the detailed features of small targets such as small buildings.

6. The building extraction method based on high-resolution networks according to claim 1, characterized in that, The model training process divides the augmented dataset into training, validation, and test sets, trains the constructed ROS-HRNet model using the Adam optimizer, employs a cosine decay learning rate strategy, and uses a binary cross-entropy loss function. The training process includes forward propagation, loss calculation, and backpropagation until the model converges.

Images