A remote sensing image building extraction method and system based on strip convolution and structure perception mask

Abstract

The application provides a remote sensing image building extraction method and system based on strip convolution and structure perception mask, and belongs to the technical field of remote sensing image processing. The method is executed by a neural network model, and comprises the following steps: inputting a remote sensing image into an encoder of the neural network model, adopting strip convolution to extract multi-scale features, so as to enhance the perception ability of long-distance spatial dependence characteristics of buildings in the remote sensing image; an decoder fuses the encoder features and the decoder features through a structure perception mask; and outputting a building segmentation result according to final fusion features output by the decoder. The system comprises a feature extraction module, a feature fusion and reconstruction module and a result generation module. Through the complementary mechanism design of strip convolution and structure perception multi-scale mask, the application effectively captures the long-distance spatial dependence characteristics of buildings, solves the limitations of traditional square convolution kernels, optimizes the feature fusion in skip connection, filters redundant information, and enhances boundary details.

Description

Technical Field

[0001] This invention relates to the field of remote sensing image processing technology, and in particular to a method and system for extracting buildings from remote sensing images based on strip convolution and structure-aware masking. Background Technology

[0002] Building extraction from remote sensing images is a crucial task in applications such as urban planning, disaster assessment, and geographic information system (GIS) construction. With the development of deep learning technology, image semantic segmentation methods based on convolutional neural networks (CNNs) have become mainstream in this field. Among them, UNet and its variants, due to their encoder-decoder structure and skip connections, can effectively fuse shallow details with deep semantic information, achieving significant success in segmentation tasks such as remote sensing images.

[0003] However, directly applying the traditional UNet architecture to extract buildings from high-resolution, complex-background remote sensing images still faces a series of challenges: 1. Building targets have diverse shapes, among which slender buildings such as bridges and large factories have significant aspect ratio characteristics. The square convolutional kernels (such as 3×3, 5×5) used in traditional UNet encoders have fixed receptive field shapes, making it difficult to effectively model the long-distance dependencies of such targets. This results in insufficient feature representation and is prone to breakage or missed detection in the segmentation results.

[0004] 2. The standard UNet uses skip connections to directly concatenate the low-level features of the encoder (rich in detail but weak in semantics) with the high-level features of the decoder (strong in semantics but low in resolution). This simple operation transmits a large amount of redundant background information contained in the encoder features to the decoder, which not only increases the computational burden of subsequent layers, but also interferes with the decoder's understanding and reconstruction of the target semantic information, resulting in blurred and inaccurate boundary segmentation.

[0005] 3. Accurate segmentation of building edges is crucial for practical applications. Existing methods lack targeted optimization for boundaries, and the segmentation results are prone to jagged edges, burrs, or discontinuities at building edges, making it difficult to meet the needs of high-precision mapping.

[0006] 4. To improve performance, subsequent studies have introduced complex attention mechanisms (such as channel attention and spatial attention) into UNet. While these mechanisms can improve feature selection capabilities to some extent, they also significantly increase the number of model parameters and computational complexity, making them unsuitable for deployment on edge devices with limited computing resources or in scenarios requiring real-time processing.

[0007] To overcome these shortcomings, those skilled in the art have been seeking improvements, such as designing more efficient convolutional operations, optimizing feature fusion methods for skip connections, or introducing more refined loss functions. However, how to systematically and collaboratively solve a series of interrelated problems—feature capture of slender targets, redundant filtering of skip connections, and refined boundary segmentation—while maintaining the model's lightweight nature, remains a challenge in current remote sensing image building extraction techniques. Summary of the Invention

[0008] This invention aims to provide a method and system for extracting buildings from remote sensing images based on strip convolution and structure-aware masks. By designing a complementary mechanism of strip convolution and structure-aware multi-scale masks, it effectively captures the long-distance spatial dependence characteristics of buildings, overcomes the limitations of traditional square convolution kernels, and optimizes feature fusion in skip connections through multi-scale feature extraction and structure-aware masks, filtering redundant information and enhancing boundary details.

[0009] The technical solution adopted in this invention is: A method for extracting buildings from remote sensing images based on strip convolution and structure-aware masking, executed by a neural network model, includes the following steps: Step S1: Input the remote sensing image into the encoder of the neural network model to extract multi-scale features of the remote sensing image; the encoder performs strip convolution operation using a strip convolution kernel in at least one feature extraction stage; wherein the size of the strip convolution kernel in the first dimension and the size in the second dimension perpendicular to it are different, so as to enhance the perception ability of the long-distance spatial dependence characteristics of buildings in the remote sensing image. Step S2: The multi-size features output by the encoder are upsampled by the decoder of the neural network model; in at least one upsampling stage, the decoder obtains the encoder features upsampled to the current resolution and the decoder features obtained from the corresponding layer of the encoder, generates a structure-aware mask based on the encoder features and the decoder features, and uses the structure-aware mask to fuse the encoder features and the decoder features. Step S3: Based on the final fusion features output by the decoder, generate and output the building segmentation result of the remote sensing image.

[0010] Furthermore, the neural network model is trained through the following process: Obtain a training dataset, which contains multiple pairs of remote sensing images and corresponding building segmentation ground truth labels; The remote sensing images in the training dataset are input into an initial neural network model to be trained to obtain the predicted segmentation results; Calculate the joint loss function value, which includes bce loss, dice loss, sdf loss, and band loss, expressed as: L = Lbce + Ldice + λ1 × Lsdf + λ2 × Lband; where L, Lbce, Ldice, Lsdf, and Lband represent the joint loss function, bce loss, dice loss, sdf loss, and band loss, respectively, and λ1 and λ2 represent the weights of sdf loss and band loss, respectively, with recommended values ​​of 0.3 and 0.05. Based on the joint loss function, the parameters of the initial neural network model are adjusted until the model converges.

[0011] Furthermore, in step S1, the following steps are performed before inputting the remote sensing image into the encoder: The remote sensing images are standardized and resized. And / or, in step S3, generating and outputting the building segmentation result of the remote sensing image based on the final fusion features output by the decoder specifically includes: The final fused features are upsampled, and then a 4×4 transposed convolution is used to map the upsampled features to the original image size. The features are refined using a 3×3 convolution and a ReLU activation function, and a segmentation mask is generated using a 1×1 convolution and a Sigmoid activation function. The segmentation mask is thresholded to obtain a binary segmentation result, which is the building segmentation result of the input remote sensing image.

[0012] Furthermore, in step S1, the encoder includes multiple sequentially connected feature extraction stages, and an attention block is connected after at least one feature extraction stage, or an attention block is connected after each feature extraction stage. The strip attention block is configured to: receive the features output from the current feature extraction stage as input features, and sequentially perform 1×1 convolutional channel projection transformation, GELU activation function processing, and strip convolution operation on the input features, and then add the processing results to the input features through residual connection and output them.

[0013] Further, in step S1, when the attention module performs the strip convolution operation, it includes: The local spatial information of the input features received by the strip attention block is extracted using a 5×5 depth convolution. Use 1×K1 horizontal strip convolution to capture the horizontal long-range dependency of the input features received by the strip attention block; where K1 is an integer greater than 10, and a recommended value is 19~31; K2×1 vertical strip convolution is used to capture the long-range vertical dependence of the input features received by the strip attention block; where K2 is an integer greater than 10, the recommended value is 19~31, and K2=K1 is preferred; The local spatial information, the long-range horizontal dependency, and the long-range vertical dependency are fused using 1×1 convolutions to generate a spatial attention map; The spatial attention map is multiplied element-wise with the input features received by the strip attention block to achieve feature recalibration and complete the strip convolution operation.

[0014] Further, in step S2, the process of generating a structure-aware mask based on the encoder features and the decoder features, and fusing the encoder features and decoder features using the structure-aware mask, includes: Feature projection is performed on the encoder features and the decoder features respectively; Based on the projected features, extract multi-scale contextual features; Based on the multi-scale contextual features, a structure-aware mask is generated, and the encoder features are modulated using the structure-aware mask. The modulated features are initially fused with the decoder features; The features after initial fusion are further fused, and the final fused features are output.

[0015] Furthermore, in step S2, 1×1 convolution is used when projecting the encoder features and the decoder features respectively; And / or, in step S2, dilated convolution is used when initially fusing the modulated features with the decoder features; And / or, in step S2, a 1×1 convolution is used when further fusing the initially fused features.

[0016] Furthermore, in step S2, extracting multi-scale contextual features based on the projected features includes: The first stage uses 3×3 depthwise convolution and 5×5 depthwise convolution to extract multi-scale features from the projected features; The second stage uses 3×3 depth convolution and 5×5 depth convolution to extract multi-scale features again from the multi-size features extracted in the first stage. Use 1×1 convolution to fuse the multi-scale features extracted in the second stage; The multi-scale context features are obtained by normalizing the channel dimensions using layerNorm.

[0017] Further, in step S2, generating a structure-aware mask based on the multi-scale context features and modulating the encoder features using the structure-aware mask includes: Different scale masks are generated using 1×1, 3×3, and 5×5 depthwise convolutions for the multi-scale contextual features. The structure-aware mask is obtained by adding masks of different scales together and then activating them using the Softmax function. The structure-aware mask is multiplied element-wise with the encoder features; The features after masking are added together to complete the modulation.

[0018] Based on the same inventive concept, this invention also provides a remote sensing image building extraction system based on strip convolution and structure-aware masking, for implementing the aforementioned remote sensing image building extraction method based on strip convolution and structure-aware masking, including: The feature extraction module is configured to input an input remote sensing image into an encoder of a neural network model to extract multi-scale features of the remote sensing image; and to perform strip convolution operation using a strip convolution kernel in at least one feature extraction stage of the encoder; wherein the size of the strip convolution kernel in the first dimension and the size in the second dimension perpendicular to it are different, so as to enhance the perception of the long-distance spatial dependence characteristics of buildings in the remote sensing image. The feature fusion and reconstruction module is configured to upsample the multi-scale features output by the encoder through the decoder of the neural network model to reconstruct the segmentation result; the decoder, in at least one upsampling stage, acquires encoder features upsampled to the current resolution and decoder features acquired from the corresponding layer of the encoder, generates a structure-aware mask based on the encoder features and the decoder features, and uses the structure-aware mask to fuse the encoder features and the decoder features; The result generation module is configured to generate and output building segmentation results of the remote sensing image based on the features finally output by the decoder.

[0019] The beneficial effects of this invention are: 1. This invention introduces a "strip convolution kernel" with a size significantly larger than another dimension in at least one dimension to perform strip convolution operations. This kernel can expand the receptive field along a specific direction (horizontal or vertical), thereby effectively modeling the long-distance spatial dependencies unique to slender structures such as bridges, roads, and ships. This avoids the problem of target feature dilution caused by traditional square convolution kernels covering too much background, significantly improving the feature representation ability and segmentation accuracy for such targets. Simultaneously, this invention innovatively introduces a "structure-aware mask" in the feature fusion stage. This mask is dynamically generated based on encoder and decoder features, adaptively identifying and enhancing structurally significant regions (such as building boundaries) from encoder features while suppressing redundant background information. This "generation-modulation-fusion" mechanism replaces the simple feature stitching in standard UNet, achieving intelligent filtering and optimization of skip connection information. This significantly improves the final segmentation results, especially the clarity, continuity, and accuracy of building boundaries, while reducing computational interference. Furthermore, this invention integrates "strip convolution" and "structure-aware masking" into a unified encoder-decoder framework. The former enhances feature extraction during the encoding stage, while the latter optimizes feature fusion during the decoding stage. The two work together to address the challenges of complex remote sensing scenarios, and achieve performance improvement without introducing a complex global attention mechanism. This maintains the relative simplicity of the model architecture, helps control computational complexity, and makes deployment possible in resource-constrained environments.

[0020] 2. The neural network model training method in this invention, particularly the joint boundary supervision loss function comprising BCE loss, DICE loss, SDF loss, and Band loss, provides precise guidance for model optimization. Specifically, BCE and DICE losses jointly supervise the overall accuracy of the segmented region; SDF loss directly utilizes the signed distance field to geometrically constrain the boundary position; and Band loss automatically focuses on the boundary region for weighted optimization. This combination achieves multi-level and refined supervision of the overall building region, boundary geometry, and boundary details. Furthermore, the introduction of SDF and Band losses is an effective solution specifically addressing the problem of "limited boundary segmentation accuracy," forcing the model to highly value the prediction of boundary pixels during the learning process. This significantly reduces jagged edges, spikes, and discontinuities in the segmentation results, resulting in smoother and more accurate building outlines. Moreover, by reasonably setting the loss weights (λ1, λ2), this joint loss function can balance different supervision objectives, guiding model parameters towards the global optimum, improving the stability of the training process and the model's final convergence performance.

[0021] 3. By standardizing and resizing the input remote sensing images, this invention eliminates the differences in scale and numerical distribution between different source images, providing a stable and standardized input for subsequent neural network processing. This helps to improve the convergence speed, generalization ability, and robustness of the model training in practical applications.

[0022] 4. This invention provides a flexibly configurable performance enhancement module for the model by integrating a "strip attention block" after the encoder feature extraction stage. This module, through a residual connection structure, introduces the enhancement capability of strip convolution while ensuring smooth gradient flow, avoiding potential performance degradation or training difficulties caused by network deepening. Simultaneously, the strip attention block uses 1×1 convolution for channel projection and combines it with GELU activation, effectively controlling parameter growth while increasing nonlinear expressive power, thus meeting the pursuit of model efficiency in the background technology.

[0023] 5. This invention achieves progressive feature modeling from local details to directional global context through "local context convolution + horizontal strip convolution + vertical strip convolution," resulting in more comprehensive information extraction. Simultaneously, by generating a "spatial attention map" and "recalibrating" the original input features, this mechanism adaptively highlights spatial regions in the input features that are important to the current task, achieving dynamic feature selection and enhancement, and further improving feature quality.

[0024] 6. This invention defines the fusion process as a standardized process of "projection → multi-scale extraction → mask generation and modulation → preliminary fusion → further fusion", making the internal working mechanism of this innovative module clear and explainable. This process shows that fusion is not a simple operation, but a deep, multi-stage interaction and processing of encoder and decoder features, ensuring that the final fused features are rich in multi-scale contextual information and finely modulated structural information.

[0025] 7. This invention uses 1×1 convolution for projection and further fusion, which is an efficient way to integrate channel information; introducing dilated convolution for preliminary fusion can expand the receptive field during fusion without increasing the number of parameters, which helps to capture a wider range of contextual information, thereby improving the representational ability of fused features.

[0026] 8. This invention, through two-stage, multi-scale convolutional operations, can repeatedly extract and refine features from different receptive field levels. The generated features contain extremely rich multi-scale contextual information, laying a solid foundation for the subsequent generation of high-quality structure-aware masks. Simultaneously, normalizing the channel dimensions using LayerNorm helps stabilize the training process, accelerate convergence, and improve the model's generalization performance.

[0027] 9. This invention employs parallel generation of multi-scale masks using 1×1, 3×3, and 5×5 depthwise convolutions, enabling simultaneous perception of structural saliency at different scales. This ensures the generated masks exhibit good response to building structures of varying sizes. Furthermore, the Softmax function fuses and activates the multi-scale masks, ensuring the generated mask values ​​remain within a reasonable range (e.g., probabilistic or between 0 and 1), making them more stable and effective as modulation weights. Moreover, the element-wise multiplication modulation method achieves fine-grained, pixel-level weighting of encoder features in the spatial dimension, truly "enhancing important structural features and suppressing redundant information." Attached Figure Description

[0028] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 This is the backbone structure of the improved ResNet50-UNet network in Example 1.

[0030] Figure 2 This is a schematic diagram of the process of extracting buildings from remote sensing images based on the improved ResNet50-UNet network in Example 1. Detailed Implementation

[0031] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," and "circumferential" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0032] The following disclosure provides many different embodiments or examples for implementing different structures of the present invention. To simplify the disclosure of the present invention, the components and arrangements of specific examples are described below. Of course, these are merely examples and are not intended to limit the present invention.

[0033] The embodiments of the invention will now be described in detail with reference to the accompanying drawings.

[0034] Example 1

[0035] To address the challenge of systematically and collaboratively solving a series of interrelated problems in current remote sensing image building extraction techniques—such as capturing slender target features, filtering redundant skip connections, and refining boundary segmentation—while maintaining a lightweight model, the applicant proposes a remote sensing image building extraction method based on strip convolution and structure-aware masking. This method is executed by a neural network model and includes the following steps: Step S1: Input the remote sensing image into the encoder of the neural network model to extract multi-scale features of the remote sensing image; the encoder performs strip convolution operation using a strip convolution kernel in at least one feature extraction stage; wherein the size of the strip convolution kernel in the first dimension and the size in the second dimension perpendicular to it are different, so as to enhance the perception ability of the long-distance spatial dependence characteristics of buildings in the remote sensing image. Step S2: The multi-size features output by the encoder are upsampled by the decoder of the neural network model; in at least one upsampling stage, the decoder obtains the encoder features upsampled to the current resolution and the decoder features obtained from the corresponding layer of the encoder, generates a structure-aware mask based on the encoder features and the decoder features, and uses the structure-aware mask to fuse the encoder features and the decoder features. Step S3: Based on the final fusion features output by the decoder, generate and output the building segmentation result of the remote sensing image.

[0036] The neural network model is trained through the following process: A training dataset is obtained, which contains multiple pairs of remote sensing images and corresponding building segmentation real labels; subsequent segmentation results generated by the neural network model are all included in the training dataset for model optimization. The remote sensing images in the training dataset are input into an initial neural network model to be trained to obtain the predicted segmentation results; The joint loss function value is calculated, which includes BCE loss (binary cross-entropy loss, essentially a pixel-level binary classification loss; used to supervise the overall segmentation region, measuring the difference between the model's predicted probability and the true label), DICE loss (measuring the consistency between the predicted result and the true label from the perspective of region overlap, used to handle class imbalance problems and enhance the detection of small objects), SDF loss (providing spatial geometric information, based on the loss of the signed distance field, supervising the prediction of boundary positions), and band loss (boundary-weighted loss, enhancing attention to boundary regions, automatically forming a "narrow boundary band" through SDF information, without the need for separate boundary extraction), which is expressed as: L=L bce +Ldice +λ1×L sdf +λ2×L band ; ; ; ; ; Among them, L, L bce L dice L sdf L band These represent the joint loss function, BCE loss, DICE loss, SDF loss, and Band loss, respectively. λ1 and λ2 represent the weights of the SDF loss and Band loss, respectively, and the recommended value for both is 0.2; Y(i) represents the true label (0 or 1); P(i) represents the prediction probability; N represents the total number of pixels; ε is a positive number to prevent the denominator from being zero, for example, e. -6 ; D(i) represents spatial geometric information: inside the building: D(i) < 0, outside the building: D(i) > 0, boundary: D(i) = 0, truncate [-20, 20] and normalize to [-1, 1]; W(i) represents the boundary weight, W(i) = 1 - |D(i)|. Near the boundary: D≈0 → W≈1 (large weight), far from the boundary: |D|≈1 → W≈0 (small weight). Based on the joint loss function, the gradient is calculated through backpropagation, the initial neural network model parameters are updated using the AdamW optimizer, and cosine annealing learning rate scheduling is applied until the model converges.

[0037] The beneficial effects of the above technical solution are as follows: This invention introduces a "strip convolution kernel" with a size significantly larger than that of another dimension to perform strip convolution operations. This kernel can expand the receptive field along a specific direction (horizontal or vertical), thereby effectively modeling the long-distance spatial dependencies unique to slender structures such as bridges and roads. This avoids the problem of target feature dilution caused by traditional square convolution kernels covering too much background, significantly improving the feature representation ability and segmentation accuracy for such targets. Simultaneously, this invention innovatively introduces a "structure-aware mask" in the feature fusion stage. This mask is dynamically generated based on encoder and decoder features, adaptively identifying and enhancing structurally significant regions (such as building boundaries) from encoder features while suppressing redundant background information. This "generation-modulation-fusion" mechanism replaces the simple feature stitching in standard UNet, achieving intelligent filtering and optimization of skip connection information. This significantly improves the final segmentation results, especially the clarity, continuity, and accuracy of building boundaries, while reducing computational interference. Furthermore, this invention integrates "strip convolution" and "structure-aware masking" into a unified encoder-decoder framework. The former enhances feature extraction during the encoding stage, while the latter optimizes feature fusion during the decoding stage. The two work together to address the challenges of complex remote sensing scenarios, and achieve performance improvement without introducing a complex global attention mechanism. This maintains the relative simplicity of the model architecture, helps control computational complexity, and makes deployment possible in resource-constrained environments.

[0038] The following is a more specific example to illustrate this.

[0039] A UNet network is constructed based on a ResNet50 encoder (each feature extraction stage of the encoder is followed by an attention block to enhance feature representation). This network is combined with striped convolutions, and skip connections are optimized using structure-aware masks to achieve end-to-end building extraction. The improved ResNet50-UNet network backbone structure is as follows: Figure 1 As shown, the improved ResNet50-UNet network is trained using the aforementioned model training method. Figure 2 As shown, the process of extracting buildings from remote sensing images based on the improved ResNet50-UNet network is as follows: 1) Input preprocessing steps: Adjust the remote sensing image to 256×256 size and perform standardization; then input it into the ResNet50 encoder; 2) Feature Encoding Step: The strip attention block in the ResNet50 encoder receives the features output from the current feature extraction stage as input features, and sequentially performs 1×1 convolutional channel projection transformation, GELU activation function processing, and strip convolution operation on the input features. The processing result is then added to the input features through residual connections and output, ultimately generating features at four scales: e1 (256 channels), e2 (512 channels), e3 (1024 channels), and e4 (2048 channels). The strip convolution operation includes using 5×5 depthwise convolution. Local spatial information of the input features received by the strip attention block is extracted; a 1×25 horizontal strip convolution is used to capture the horizontal long-range dependency of the input features received by the strip attention block; a 25×1 vertical strip convolution is used to capture the vertical long-range dependency of the input features received by the strip attention block; the local spatial information, horizontal long-range dependency, and vertical long-range dependency are fused using a 1×1 convolution to generate a spatial attention map; the spatial attention map is multiplied element-wise with the input features received by the strip attention block to achieve feature recalibration and complete the strip convolution operation.

[0040] 3) Feature decoding steps: The e4 feature is upsampled by the DecoderBlock of the decoder to obtain the d4 feature. Then, the d4 feature is fused with the e3 feature to obtain the optimized d4' feature. The fusion process includes: projecting e4 and d4 features using 1×1 convolutions; extracting multi-scale context features based on the projected features; in the first stage, extracting multi-scale features from the projected features using 3×3 and 5×5 depth convolutions; in the second stage, extracting multi-scale features again from the multi-scale features extracted in the first stage using 3×3 and 5×5 depth convolutions; fusing the multi-scale features extracted in the second stage using 1×1 convolutions; normalizing the channel dimensions using layerNorm to obtain multi-scale context features; generating masks of different scales using 1×1, 3×3, and 5×5 depth convolutions for the multi-scale context features; adding the masks of different scales and activating them using the Sigmoid function to obtain a structure-aware mask; multiplying the structure-aware mask element-wise with the e4 features; adding the masked features to complete modulation; performing preliminary fusion of the modulated features and the d4 features using dilated convolutions; further fusion processing using 1×1 convolutions on the preliminarily fused features to output the d4' feature.

[0041] The d4' feature is upsampled by the DecoderBlock of the decoder to obtain d3. Then, following the aforementioned fusion process, the d3 feature is fused with the e2 feature to obtain the optimized d3' feature.

[0042] The d3' feature is upsampled by the DecoderBlock of the decoder to obtain the d2 feature. Then, following the aforementioned fusion process, the d2 feature is fused with the e1 feature to obtain the optimized d2' feature.

[0043] The d2' feature is upsampled by the DecoderBlock of the decoder to obtain the d1 feature.

[0044] 4) Output Steps: The d1 features are finally upsampled, then a 4×4 transposed convolution is used to map the features to the original image size. A 3×3 convolution and ReLU activation function are used to refine the features. A 1×1 convolution and Sigmoid activation function are used to generate a segmentation mask. The segmentation mask is then thresholded to 0.5 to obtain the binary segmentation result. The remote sensing image and its corresponding segmentation result are included in the training dataset of the ResNet50-UNet neural network model for model optimization.

[0045] The method for extracting buildings from remote sensing images based on this ResNet50-UNet neural network model has the following advantages: 1. Traditional square convolution kernels struggle to effectively capture long-range dependencies in slender buildings, resulting in low segmentation accuracy for slender targets such as bridges and roads. This method, through strip convolution modules, captures long-range dependencies in both horizontal and vertical directions, significantly improving the extraction accuracy of slender buildings. Experimental results show that the IoU is improved by approximately 0.5-1.2% compared to the control method.

[0046] 2. Traditional skip connections directly concatenate features, introducing redundant information and resulting in blurred and discontinuous boundary segmentation. This method enhances the ability to capture boundary details through a structure-aware multi-scale masking module and joint boundary supervision loss, improving boundary IoU by approximately 6.3-7.1% and producing clearer and more continuous segmentation results.

[0047] 3. While traditional complex attention mechanisms can improve performance, they significantly increase computational complexity, which is not conducive to real-time applications. This method employs lightweight designs such as depthwise convolution and grouped convolution, which controls the model's computational complexity while ensuring segmentation accuracy.

[0048] 4. Traditional methods struggle to adapt to building scenes of varying scales and complexities, performing poorly in dense or occluded environments. This method, through multi-scale feature extraction and structure-aware masking, enables the model to adapt to building scenes of different scales and complexities, improving segmentation accuracy in dense building regions by approximately 0.8-2.4%.

[0049] 5. Traditional models are sensitive to noise and illumination changes and have poor robustness. This method, through strip convolution and structure-aware masking, significantly improves the model's robustness to noise and illumination changes, and performs more stably in complex remote sensing scenarios.

[0050] 6. Traditional complex models suffer from slow inference speeds, making them unsuitable for real-time applications. The lightweight design and optimized computational flow of this method enable the model to achieve real-time inference on ordinary GPUs, supporting online monitoring and real-time analysis.

[0051] 7. Traditional models have limited overall performance in building extraction tasks. This method comprehensively utilizes strip convolution, structure-aware multi-scale masking, and joint boundary supervision loss, significantly improving overall segmentation performance. On the WHU Aerial imagery dataset, the IoU reaches 87.68%, exceeding the control method by approximately 0.4-1.6%.

[0052] 8. By integrating multiple innovative technologies, this method significantly improves the performance of building extraction from remote sensing images, providing reliable technical support for applications such as urban planning, disaster assessment, and land use monitoring, and has significant practical application value.

[0053] Example 2 This embodiment provides a remote sensing image building extraction system based on strip convolution and structure-aware masking, used to implement the remote sensing image building extraction method based on strip convolution and structure-aware masking as described in Embodiment 1, including: The feature extraction module is configured to input an input remote sensing image into an encoder of a neural network model to extract multi-scale features of the remote sensing image; and to perform strip convolution operation using a strip convolution kernel in at least one feature extraction stage of the encoder; wherein the size of the strip convolution kernel in the first dimension and the size in the second dimension perpendicular to it are different, so as to enhance the perception of the long-distance spatial dependence characteristics of buildings in the remote sensing image. The feature fusion and reconstruction module is configured to upsample the multi-scale features output by the encoder through the decoder of the neural network model to reconstruct the segmentation result; the decoder, in at least one upsampling stage, acquires encoder features upsampled to the current resolution and decoder features acquired from the corresponding layer of the encoder, generates a structure-aware mask based on the encoder features and the decoder features, and uses the structure-aware mask to fuse the encoder features and the decoder features; The result generation module is configured to generate and output building segmentation results of the remote sensing image based on the features finally output by the decoder.

[0054] The following will illustrate this with more specific examples.

[0055] A remote sensing image building extraction system based on striped convolution and structure-aware masking was deployed on the target hardware platform. The remote sensing image building extraction method based on striped convolution and structure-aware masking, as described in Example 1, was implemented through loading and running, completing the system integration and functional implementation. The corresponding test results are as follows: 【1】Model training phase Hardware configuration: GPU: NVIDIA GeForce RTX 5060 Laptop GPU, 8GB VRAM CPU: Intel® Core™ Ultra 9 275HX, 24 cores Training parameters: Batch size: 4 Learning rate: 1e -3 (AdamW optimizer) Learning rate scheduling: cosine annealing Weight decay: 5e -4 Number of training rounds: 200 Early Stop Patience Value: 20 Minimum number of training rounds: 70 Mixed precision training: FP16 + FP32 λ1: 0.3 λ2: 0.05 Training data: Dataset: WHU Aerial imagery dataset (4736 images for training + 1036 images for evaluation) Input dimensions: 512×512 Data augmentation: random flipping, rotation, scaling Training steps: 1. Data preprocessing: Adjust the image and mask to 256×256 size and perform standardization.

[0056] 2. Model initialization: Load the pre-trained ResNet50 weights.

[0057] 3. Training cycle: Forward propagation: Calculates the model output and loss.

[0058] Loss function: BCE + Dice + SDF + Band loss.

[0059] Backpropagation: Calculates the gradient and updates the model parameters.

[0060] Validation: Validate once every epoch and save the best model.

[0061] Training results: IoU ratio: 86.45%; Boundary IoU: 68.32%.

[0062] [2] Model reasoning stage Hardware configuration: GPU: NVIDIA GeForce RTX 5060 Laptop GPU, 8GB VRAM CPU: Intel® Core™ Ultra 9 275HX, 24 cores Inference parameters: Batch size: 4 Test data: Dataset: WHU Aerial imagery dataset (test 2416 images) Input dimensions: 512×512 Reasoning process: 1. Input image preprocessing: Adjust to 256×256 size and standardize.

[0063] 2. Model Inference: Input the preprocessed image to obtain the segmentation mask.

[0064] 3. Post-processing: Apply a threshold of 0.5 to the segmentation mask to obtain the binary segmentation result.

[0065] 4. Output Results: Save the segmentation results or display them visually.

[0066] Inference performance: Intersection over union (IoU) ratio: 87.68%; Boundary Intersection Unification (IoU): 69.59%; Precision: 91.93%; Recall rate: 91.64%; F1 Score: 91.79%; Inference time for a single 512×512 image: approximately 3ms (GPU).

[0067] The above example illustrates that the remote sensing image building extraction system based on strip convolution and structure-aware mask in this embodiment, after deployment, has the characteristics of high accuracy (especially good at boundary segmentation), high efficiency (meets real-time requirements), high practicality (easy to deploy, complete process) and good robustness.

Claims

1. A method for extracting buildings from remote sensing images based on strip convolution and structure-aware masking, characterized in that, Performed by a neural network model, the process includes the following steps: Step S1: Input the remote sensing image into the encoder of the neural network model to extract multi-scale features of the remote sensing image; the encoder performs strip convolution operation using a strip convolution kernel in at least one feature extraction stage; wherein the size of the strip convolution kernel in the first dimension and the size in the second dimension perpendicular to it are different, so as to enhance the perception ability of the long-distance spatial dependence characteristics of buildings in the remote sensing image. Step S2: The multi-size features output by the encoder are upsampled by the decoder of the neural network model; in at least one upsampling stage, the decoder obtains the encoder features upsampled to the current resolution and the decoder features obtained from the corresponding layer of the encoder, generates a structure-aware mask based on the encoder features and the decoder features, and uses the structure-aware mask to fuse the encoder features and the decoder features. Step S3: Based on the final fusion features output by the decoder, generate and output the building segmentation result of the remote sensing image.

2. The method for extracting buildings from remote sensing images based on strip convolution and structure-aware masking according to claim 1, characterized in that, The neural network model is trained through the following process: Obtain a training dataset, which contains multiple pairs of remote sensing images and corresponding building segmentation real labels; The remote sensing images in the training dataset are input into an initial neural network model to be trained to obtain the predicted segmentation results; Calculate the joint loss function value, which includes BCE loss, DICE loss, SDF loss, and Band loss, expressed as: L = L bce +L dice +λ1×L sdf +λ2×L band ; Among them, L, L bce L dice L sdf L band Let represent the joint loss function, BCE loss, DICE loss, SDF loss, and Band loss, respectively; λ1 and λ2 represent the weights of the SDF loss and Band loss, respectively. Based on the joint loss function, the parameters of the initial neural network model are adjusted until the model converges.

3. The method for extracting buildings from remote sensing images based on strip convolution and structure-aware masking according to claim 1, characterized in that, In step S1, the following steps are performed before inputting the remote sensing image into the encoder: The remote sensing images are standardized and resized. And / or, in step S3, generating and outputting the building segmentation result of the remote sensing image based on the final fusion features output by the decoder specifically includes: The final fused features are upsampled, and then a 4×4 transposed convolution is used to map the upsampled features to the original image size. The features are refined using a 3×3 convolution and a ReLU activation function, and a segmentation mask is generated using a 1×1 convolution and a Sigmoid activation function. The segmentation mask is thresholded to obtain a binary segmentation result, which is the building segmentation result of the input remote sensing image.

4. The method for extracting buildings from remote sensing images based on strip convolution and structure-aware masking according to claim 1, characterized in that, In step S1, the encoder includes multiple sequentially connected feature extraction stages, and at least one feature extraction stage is followed by an attention block, or each feature extraction stage is followed by an attention block. The strip attention block is configured to: receive the features output from the current feature extraction stage as input features, and sequentially perform 1×1 convolutional channel projection transformation, GELU activation function processing, and strip convolution operation on the input features, and then add the processing results to the input features through residual connection and output them.

5. The method for extracting buildings from remote sensing images based on strip convolution and structure-aware masking according to claim 4, characterized in that, In step S1, the attention module performs strip convolution operations including: Local spatial information of the input features received by the strip attention block is extracted using a 5×5 depth convolution. The horizontal long-range dependency of the input features received by the strip attention block is captured using a 1×K1 horizontal strip convolution; where K1 is an integer greater than 10. K2×1 vertical strip convolution is used to capture the long-range vertical dependence of the input features received by the strip attention block; where K2 is an integer greater than 10. The local spatial information, the long-range horizontal dependency, and the long-range vertical dependency are fused using 1×1 convolutions to generate a spatial attention map; The spatial attention map is multiplied element-wise with the input features received by the strip attention block to achieve feature recalibration and complete the strip convolution operation.

6. The method for extracting buildings from remote sensing images based on strip convolution and structure-aware masking according to any one of claims 1 to 5, characterized in that, In step S2, the process of generating a structure-aware mask based on the encoder features and the decoder features, and then fusing the encoder features and decoder features using the structure-aware mask, includes: Feature projection is performed on the encoder features and the decoder features respectively; Based on the projected features, extract multi-scale contextual features; Based on the multi-scale contextual features, a structure-aware mask is generated, and the encoder features are modulated using the structure-aware mask. The modulated features are initially fused with the decoder features; The features after initial fusion are further fused, and the final fused features are output.

7. The method for extracting buildings from remote sensing images based on strip convolution and structure-aware masking according to claim 6, characterized in that, In step S2, 1×1 convolution is used when projecting the encoder features and the decoder features respectively. And / or, in step S2, dilated convolution is used when initially fusing the modulated features with the decoder features; And / or, in step S2, a 1×1 convolution is used when further fusing the initially fused features.

8. The method for extracting buildings from remote sensing images based on strip convolution and structure-aware masking according to claim 6, characterized in that, In step S2, extracting multi-scale context features based on the projected features includes: The first stage uses 3×3 depthwise convolution and 5×5 depthwise convolution to extract multi-scale features from the projected features; The second stage uses 3×3 depth convolution and 5×5 depth convolution to extract multi-scale features again from the multi-size features extracted in the first stage. Use 1×1 convolution to fuse the multi-scale features extracted in the second stage; The multi-scale context features are obtained by normalizing the channel dimensions using layerNorm.

9. The method for extracting buildings from remote sensing images based on strip convolution and structure-aware masking according to claim 6, characterized in that, In step S2, generating a structure-aware mask based on the multi-scale context features and modulating the encoder features using the structure-aware mask includes: Different scale masks are generated using 1×1, 3×3, and 5×5 depthwise convolutions for the multi-scale contextual features. The structure-aware mask is obtained by adding masks of different scales together and then activating them using the Softmax function. Multiply the structure-aware mask element-wise with the encoder features; The features after masking are added together to complete the modulation.

10. A remote sensing image building extraction system based on strip convolution and structure-aware masking, used to implement the remote sensing image building extraction method based on strip convolution and structure-aware masking as described in any one of claims 1 to 9, characterized in that, include: The feature extraction module is configured to input the input remote sensing image into the encoder of a neural network model to extract multi-scale features of the remote sensing image; In at least one feature extraction stage of the encoder, strip convolution operation is performed using a strip convolution kernel; wherein the size of the strip convolution kernel in the first dimension and the size in the second dimension perpendicular to it are different, so as to enhance the perception of the long-distance spatial dependence characteristics of buildings in the remote sensing image. The feature fusion and reconstruction module is configured to upsample the multi-scale features output by the encoder through the decoder of the neural network model to reconstruct the segmentation result; the decoder, in at least one upsampling stage, acquires encoder features upsampled to the current resolution and decoder features acquired from the corresponding layer of the encoder, generates a structure-aware mask based on the encoder features and the decoder features, and uses the structure-aware mask to fuse the encoder features and the decoder features; The result generation module is configured to generate and output building segmentation results of the remote sensing image based on the features finally output by the decoder.

Images