A remote sensing image building extraction method based on global and local feature fusion

Abstract

The present application relates to the technical field of remote sensing image processing, and is especially a remote sensing image building extraction method based on global and local feature fusion, comprising the following steps: a data receiving server receives and demodulates remote sensing images through a satellite downlink, and the remote sensing images are loaded into a graphic processor after being preprocessed and blocked by a CPU; the graphic processor extracts multi-scale features through a ResNet50 encoder, a multi-scale long-range aggregation module extracts local multi-scale context and global dependence through a dimension reduction hollow space pyramid pooling branch and a linear Transformer branch in parallel, and the local multi-scale context and the global dependence are fused and connected through a residual error; a decoder is up-sampled and spliced with the encoder features; an edge preserving convolution module enhances the boundary, a hybrid attention module suppresses the background, and a fusion generated segmentation mask is obtained; and deep supervision and a Focal-Dice hybrid loss are used for training. Through the multi-scale long-range aggregation module double-branch parallel fusion structure, the present application breaks through the limitations of the fixed receptive field of the traditional convolutional network, and effectively improves the multi-scale building segmentation precision.

Description

Technical Field

[0001] This invention relates to the field of remote sensing image processing technology, specifically to a method for extracting buildings from remote sensing images based on the fusion of global and local features. Background Technology

[0002] High-resolution Earth observation satellites (such as WorldView, GeoEye, and the Gaofen series) transmit remote sensing image data downlink to ground receiving stations via satellite-to-ground communication links. After demodulation, decoding, and format restoration, the raw remote sensing image data is formed. Ground data processing servers need to extract building areas from these remote sensing images to serve applications such as urban mapping, land surveys, and disaster assessment. Existing remote sensing image building extraction technologies mainly suffer from the following shortcomings: Existing extraction methods based on deep convolutional neural networks (such as FCN, U-Net, DeepLabV3+, etc.) use fixed-size convolutional kernels stacked to construct a feature pyramid in their backbone networks (such as ResNet, VGG, etc.). This means the receptive field coverage cannot dynamically adapt to the actual scale of the building. When the scale of the building to be extracted exceeds the preset receptive field range of the network design, the cross-layer correlation between high-level semantic features and low-level spatial details breaks down, leading to under-segmentation (incomplete regions) or over-segmentation (misclassifying non-building regions as buildings) in the segmentation results.

[0003] Secondly, buildings in remote sensing images often exhibit local similarities in spectral features with surrounding background elements such as roads, bare ground, shadows, and vegetation. Existing convolutional networks with encoding / decoding structures (such as U-Net and its variants) primarily rely on convolutional operations within local receptive fields, making it difficult to establish long-range dependencies between pixels at a global scale. When the spectral statistical features of the background region are similar to those of the building region, the network, relying solely on local neighborhood information, cannot effectively distinguish building boundaries from similar backgrounds, leading to misidentification of background regions as buildings. This problem is particularly pronounced in areas with high background complexity, such as urban-rural fringe areas and construction sites.

[0004] Furthermore, existing encoders (such as ResNet and VGG) progressively reduce the spatial resolution of feature maps using convolutional or pooling layers with a stride of 2 (typically a downsampling ratio of 32x) to increase the receptive field and reduce computational cost. However, downsampling inevitably loses high-frequency edge information, resulting in blurred boundaries and location shifts in building outlines after upsampling and reconstruction by the decoder. In densely built-up areas, the narrow gaps between adjacent buildings (typically only a few pixels wide) cannot be accurately reconstructed after downsampling-upsampling cycles, causing adjacent building segmentation results to stick together and become impossible to identify independently.

[0005] Finally, in remote sensing images, building pixels typically account for a low proportion of the total pixels (approximately 10%–30% in urban areas and less than 5% in suburban areas), with background pixels making up the vast majority. Existing training methods based on cross-entropy loss tend to predict all pixels as background to reduce the overall loss value when faced with extremely imbalanced positive and negative samples, resulting in small-scale buildings (such as detached houses and temporary buildings) being missed. Summary of the Invention

[0006] The purpose of this invention is to provide a method for extracting buildings from remote sensing images based on the fusion of global and local features, so as to solve the problems mentioned in the background art.

[0007] To achieve the above objectives, the present invention provides the following technical solution: A method for extracting buildings from remote sensing images based on the fusion of global and local features, using a computer device comprising a data receiving server, a computing server, and a storage server, wherein the computing server is equipped with a graphics processor, characterized by comprising the following steps: S1: The data receiving server receives the raw remote sensing image data stream through a satellite communication link, demodulates, decodes, and restores the format to generate the raw remote sensing image, and transmits it to the central processing unit of the computing server cluster via Ethernet; the central processing unit sequentially performs radiometric calibration, atmospheric correction, band registration, and fusion on the raw remote sensing image to generate a multi-channel fused image, and performs sliding window segmentation on the multi-channel fused image to generate several image blocks, performs normalization processing on each image block, and then moves it to the graphics processing unit's video memory of the computing server cluster; S2: The graphics processor inputs the normalized image patches into the encoder backbone network. The encoder backbone network adopts the ResNet50 architecture and extracts and outputs at least three intermediate feature maps of different spatial scales through multi-stage convolution and downsampling operations. S3: The graphics processor performs multi-scale long-range aggregation processing on each intermediate feature map. The multi-scale long-range aggregation processing includes: The first branch processing: After channel dimensionality reduction of the input feature map, multiple dilated convolutions with different dilation rates are executed in parallel. The outputs of each dilated convolution are concatenated along the channel dimension to obtain a local multi-scale context feature map. The second branch processing: After channel dimensionality reduction of the input feature map, the feature map is flattened into a sequence matrix, linear self-attention operation is performed, global long-range dependencies between pixels are established, and a global context feature map is obtained. The local multi-scale context feature map is fused with the global context feature map and compressed to the number of channels of the input feature map. The residual is then superimposed with the input feature map to output a multi-scale enhanced feature map. S4. The graphics processor inputs the multi-scale enhanced feature maps output in step 3 into the decoder, performs upsampling operations in sequence, and performs cross-layer concatenation and convolution fusion of the upsampled feature maps with the corresponding multi-scale enhanced feature maps, and outputs the final-level fused feature map of the decoder. S5: The graphics processor performs edge-preserving convolution processing on the final-level fusion feature map of the decoder, including: extracting the edge response intensity map of the final-level fusion feature map of the decoder and converting it into an edge space attention weight map, while extracting the main segmentation feature map, and performing element-wise weighted fusion of the main segmentation feature map and the edge space attention weight map to output the edge enhancement feature map. S6: The graphics processor performs hybrid attention processing on the final-stage fused feature map of the decoder, including: sequentially performing channel attention weighting and spatial attention weighting, filtering out background noise response, and outputting attention-enhanced feature map; S7: The graphics processor performs channel fusion of the edge enhancement feature map and the attention enhancement feature map, and then performs convolution mapping to obtain the pixel classification score map. After upsampling the pixel classification score map to the original size of the image block, it is segmented by activation function and threshold to generate building segmentation mask. S8: The graphics processor sends the building segmentation mask back to the central processing unit (CPU). The CPU stitches the segmentation masks of each image block according to the sliding window segmentation position to generate a complete segmentation mask that matches the size of the original remote sensing image, and outputs it to the terminal display device and storage server.

[0008] Preferably, the band registration and fusion in step S1 specifically includes: using the phase correlation method to detect the offset between the panchromatic band and the multispectral band and performing registration, using the Gram-Schmidt fusion algorithm or the principal component analysis fusion algorithm to inject panchromatic band details into the multispectral band, and generating the multichannel fused image.

[0009] Preferably, the sliding window segmentation in step S1 uses a fixed-size sliding window, and adjacent image blocks have overlapping areas in both the row and column directions.

[0010] Preferably, in step S3, the channel dimensionality reduction of the first branch process compresses the number of input feature map channels to a first preset number of channels, and the dilation rates of the multiple dilated convolutions with different dilation rates are 1, 6, 12, and 18, respectively.

[0011] Preferably, in step S3, the channel dimensionality reduction of the second branch process compresses the number of input feature map channels to a second preset number of channels, and the linear self-attention operation uses a kernel function. By performing feature mapping on the query matrix and key matrix, the computational complexity of self-attention is reduced to the linear order of the sequence length.

[0012] Preferably, the extraction of the edge response intensity map in step S5 specifically includes: using the Sobel operator to calculate the gradient response in the horizontal direction and the gradient response in the vertical direction respectively, superimposing the gradient responses in the two directions and normalizing them by the S-shaped activation function to generate the edge spatial attention weight map.

[0013] Preferably, the element-wise weighted fusion in step S5 adopts the formula ,in Main segmentation feature map For the attention weight map in the edge space, For learnable scaling parameters.

[0014] Preferably, the channel attention weighting in step S6 includes: performing global max pooling and global average pooling on the input feature map respectively, adding them after mapping through a shared multilayer perceptron and activating them to generate channel attention weights, and multiplying them element-wise with the input feature map; the spatial attention weighting includes: performing global max pooling and global average pooling on the channel-weighted feature map respectively, concatenating them and generating spatial attention weights through convolution and activation, and multiplying them element-wise with the channel-weighted feature map.

[0015] Preferably, the training steps of step S2 are as follows: auxiliary classifiers are set at the output of at least three intermediate stages of the encoder and the final output of the decoder. The output of each auxiliary classifier is upsampled and the loss is calculated with the real label. The loss of each layer is weighted and summed to obtain the total loss. The network weights are updated based on the total loss. The loss is a hybrid loss of Focal loss and Dice loss.

[0016] Preferably, the data receiving server and the computing server cluster transmit data via 10 Gigabit Ethernet and TCP / IP protocol, or via 5G mobile communication network and UDP protocol.

[0017] Compared with the prior art, the beneficial effects of the present invention are: This invention overcomes the limitations of fixed receptive fields in traditional convolutional networks by employing a dual-branch parallel fusion structure in a multi-scale long-range aggregation module, effectively improving the segmentation accuracy of multi-scale buildings. The invention utilizes a hollow spatial pyramid pooling branch to construct a multi-scale local receptive field, adapting to the extraction of detailed features from large, medium, and small-span buildings in remote sensing images. Simultaneously, a linear Transformer branch establishes global pixel long-range dependencies with linear computational complexity, compensating for the insufficient global modeling capabilities of local convolutions. Combined with a residual fusion mechanism, multi-scale local details and global semantic features are integrated, effectively solving the problems of incomplete segmentation of large-scale buildings and missed detection of small-scale buildings, significantly improving the segmentation intersection-union ratio and target recall rate.

[0018] In this invention, the false detection rate of buildings in complex backgrounds is significantly reduced through the synergistic effect of global context modeling and channel-spatial hybrid attention mechanism. Leveraging global feature interaction capabilities, this invention fully mines the unique contextual features of buildings, such as their regular outlines and spatial distribution, effectively distinguishing easily confused features like shadows, bare ground, and roads based on their spectral textures. Simultaneously, channel attention filters effective features and suppresses ineffective features, while spatial attention recalibrates pixel responses and suppresses interference from complex backgrounds, significantly improving the accuracy and robustness of building segmentation in highly complex scenarios such as urban-rural fringe areas and construction sites.

[0019] In this invention, an edge-preserving convolutional module is integrated with a Focal-Dice hybrid loss depth supervision strategy to effectively optimize building boundary segmentation and small object detection capabilities. The edge-preserving convolutional module explicitly extracts high-frequency edge features and generates adaptive attention weights to specifically enhance the response of building contour features, correcting edge information loss and boundary offset defects caused by network downsampling, accurately identifying pixel-level building gaps, and solving the problem of adjacent building segmentation adhesion. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating the remote sensing image building extraction method based on the fusion of global and local features according to the present invention. Detailed Implementation

[0021] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0022] To facilitate understanding of the present invention, a more comprehensive description of the invention will be given below with reference to the accompanying drawings, and several embodiments of the invention will be provided. However, the invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that the disclosure of the invention will be more thorough and complete.

[0023] See Figure 1 As shown, a method for extracting buildings from remote sensing images based on the fusion of global and local features is presented. This method is implemented on a hierarchical hardware platform, which includes a data receiving server, a computing server cluster, a storage server, and a terminal display device. The data receiving server is used for receiving, parsing, and initially transmitting remote sensing data; the computing server cluster is used for image preprocessing, feature extraction, feature fusion, and segmentation inference; the storage server is used for persistent storage of the dataset, model weights, and segmentation results; and the terminal display device is used for result visualization and human-computer interaction.

[0024] S1: Remote sensing image data reception and preprocessing, used to receive, interpret, correct, fuse, segment and standardize the raw satellite remote sensing data, remove imaging and transmission interference, generate regular image data that is adapted to the input of the deep learning model, and provide high-quality input for subsequent feature extraction and segmentation inference. It is specifically divided into 5 sub-steps.

[0025] Step S1.1: The data receiving server, through its supporting satellite communication receiving antenna and RF demodulation equipment, captures the raw remote sensing image data stream transmitted via the downlink from Earth observation satellites in real time. It sequentially performs carrier demodulation, LDPC low-density parity-check code decoding, frame synchronization header capture, and transmission protocol restoration on the raw data stream, outputting raw remote sensing image files conforming to GeoTIFF or NITF standards. The raw remote sensing images include panchromatic and multispectral bands, with a spatial resolution better than 1m and a pixel bit depth of 8 bits or 16 bits. After data parsing, the data receiving server transmits the image data in fragments via a high-speed Ethernet interface based on the TCP transmission control protocol, pushing the complete image data to the CPU memory of the computing server cluster.

[0026] Step S1.2: The CPU acquires the pixel quantization DN values ​​of the original remote sensing image, and performs radiometric calibration by combining them with the sensor's factory calibration parameters. The discrete pixel quantization values ​​are converted into the true surface reflectance, and the values ​​are normalized to the [0,1] interval to eliminate data deviations caused by sensor imaging gain and offset errors. The radiometric calibration algorithm formula is as follows: In the formula: For surface reflectance, This is the radiation calibration gain coefficient. This is the radiation calibration offset coefficient. This represents the quantization values ​​of the original pixels in the image.

[0027] After radiometric calibration is completed, the CPU calls the locally stored six-source radiative transfer model lookup table to perform atmospheric correction processing on the image, which cancels the scattering and absorption interference of atmospheric molecules and aerosols on the surface reflection signal, restores the true surface radiation characteristics, and improves the feature differentiation between buildings and background features.

[0028] The construction method of the six-source radiative transfer model lookup table is as follows: Based on the 6S radiative transfer model (Vermote et al., 1997), the atmospheric mode is set to mid-latitude summer or winter (selected according to the image acquisition season and geographical latitude), the aerosol mode is set to rural or urban (selected according to the image coverage area), and the aerosol optical thickness is set to five levels: 0.05, 0.1, 0.2, 0.5, and 1.0. The observed zenith angle and solar zenith angle are discretized and sampled at 5° intervals within the range of 0° to 60°. The 6S model is run offline item by item for the above parameter combinations to calculate the atmospheric correction coefficients (including atmospheric transmittance, path radiation, spherical albedo, etc.) for each combination, forming a multidimensional lookup table and storing it locally on the server. During atmospheric correction, the solar zenith angle and observation geometry are calculated based on the imaging time and latitude / longitude information recorded in the image header file. The corresponding correction coefficients are obtained by interpolation from the lookup table, and the surface reflectance is retrieved pixel by pixel.

[0029] Step S1.3: The phase correlation method is used to detect the sub-pixel offset between the panchromatic band and the multispectral band. The resampling algorithm is used to complete the accurate registration of the two bands and eliminate the parallax misalignment problem. Based on the Gram-Schmidt fusion algorithm or the principal component analysis fusion algorithm, the detailed features of the high spatial resolution panchromatic band are injected into the low resolution multispectral band to generate an R / G / B three-channel pseudo-color fused image. The spatial resolution of the fused image is uniformly calibrated to a preset fixed value (0.5m is selected in this embodiment. This value is determined before model training and remains fixed throughout the training and inference stages without being changed).

[0030] Step S1.4: The fused complete remote sensing image is divided into blocks using a fixed-size sliding window. The window size is set to 512 pixels × 512 pixels, and the row and column sliding step size is set to 448 pixels. This ensures that adjacent image blocks retain a 64-pixel overlap area in both the row and column directions, effectively avoiding boundary faults and pixel loss problems caused by subsequent image stitching, and ensuring the integrity of the whole image segmentation result.

[0031] Step S1.5: Perform zero-mean normalization on each segmented image block to eliminate pixel numerical scale differences. The normalization algorithm formula is as follows: In the formula: For standardized pixel values, These are the original pixel values. The average pixel value of a single image patch. denoted as the standard deviation of pixels in a single image patch. The standardized image data follows a standard normal distribution with a mean of 0 and a variance of 1.

[0032] After data preprocessing is complete, the CPU calls the CUDA asynchronous memory copy function to asynchronously move the normalized image block data to the GPU memory via the PCIe 4.0 or PCIe 5.0 bus, providing input data for subsequent model inference.

[0033] S2: Multi-stage downsampling feature extraction based on the ResNet50 backbone network at the encoder end: The GPU reads a normalized image tensor with dimensions [batch size, 3, 512, 512] from the GPU memory and inputs it into the pre-constructed and parameter-loaded TBH-UNet network encoder. The encoder uses ResNet50 as the core backbone network and extracts image features at different scales and semantic levels step by step through multi-stage convolution, pooling, and residual operations, specifically in five operation stages.

[0034] The first stage involves performing a 7×7 two-dimensional convolution operation with a stride of 2 on the input tensor, followed by a 3×3 max pooling operation with a stride of 2. This compresses the feature map space size to 1 / 4 of the original image, outputting a 64-channel shallow detail feature map.

[0035] The second stage involves stacking three sets of residual bottleneck modules without downsampling. The output feature map space size remains 1 / 4 of the original image, with 256 channels, preserving the basic texture and contour features of the image.

[0036] The third stage involves stacking four sets of residual bottleneck modules. The first bottleneck module uses a 3×3 convolution with a stride of 2 to achieve downsampling, and the output feature map space size is 1 / 4 of the original image with 512 channels. This is denoted as the first intermediate feature map.

[0037] Fourth stage: Stack 6 sets of residual bottleneck modules. The first bottleneck module uses a convolution with a stride of 2 to complete downsampling. The output feature map space size is 1 / 8 of the original image, and the number of channels is 1024. This is called the second intermediate feature map.

[0038] Fifth stage: Stack 3 sets of residual bottleneck modules. The first bottleneck module uses a convolution with a stride of 2 to complete downsampling. The output feature map space size is 1 / 16 of the original image, and the number of channels is 2048. This is called the third intermediate feature map.

[0039] The first, second, and third intermediate feature maps are cached in the GPU shared memory and used as input data for the subsequent multi-scale long-range aggregation module.

[0040] S3: Parallel processing and global-local feature fusion of the multi-scale long-range aggregation module: At the output of the third, fourth and fifth stages of the encoder, the multi-scale long-range aggregation (MSLRA) module is deployed in parallel. Through dual-branch parallel operation, local multi-scale contextual features and global long-range dependency features of the image are mined respectively. After fusion, enhanced features that take into account both details and semantics are obtained, which effectively solves the defects of single convolution with fixed receptive field and insufficient global modeling ability. Specifically, it is divided into 3 sub-steps.

[0041] Step S3.1: For the input feature map (dimensions [H, W, C]), firstly, a 1×1 2D convolution is used to compress the number of channels C to 256, reducing the number of subsequent computational parameters and floating-point computations. Then, four sets of 3×3 dilated convolutions are used in parallel, with dilation rates of 1, 6, 12, and 18 respectively, covering multi-scale receptive fields from near to far distances. Each set of dilated convolutions outputs 256 channels. The feature maps output by the four sets of dilated convolutions are concatenated along the channel dimensions to obtain a 1024-channel local multi-scale contextual feature map.

[0042] Step S3.2: The input feature map is reduced to 128 channels using a 1×1 convolution, flattening the two-dimensional feature map into a sequence matrix of dimension [N, 128] (N = H × W is the number of spatial words). This avoids the O(N) time complexity of traditional self-attention mechanisms. 2 Due to the extremely high computational complexity of traditional algorithms, this invention optimizes self-attention operations using the ReLU+1 linear kernel mapping function. The core algorithm formula is as follows: In the formula: For query matrix, For the key matrix, For value matrices, This is a linear kernel mapping function. This optimization reduces the computational complexity to O(N), enabling global pixel long-range dependency modeling. The resulting global features are then restored to a spatial feature map, which is then upscaled to 256 channels via a 1×1 convolution.

[0043] In the specific implementation, the query matrix Key matrix Value matrix The feature maps are generated from the 128-channel feature map using three independent 1×1 convolutional layers, each matrix having dimensions [N, 128], without employing a multi-head mechanism. A linear kernel mapping function is used. Element-wise action and Ensure that all elements are non-negative.

[0044] To avoid numerical overflow, and Add 1× to the dot product result The smallest constant As the smoothing term in the denominator, the final attention output is: Where the denominator is a scalar and the numerator is [

[128] Matrix, scalar division is performed element-wise. This design transforms traditional self-attention... Complexity reduced to ,in For spatial word elements, =128 is the feature dimension, as set in this invention. 262144 (512×512 feature map), d=128, computational cost is approximately 3.35×10 7 This is far lower than the approximately 6.87 × 10⁻⁶ of traditional self-attention. 10 .

[0045] Step S3.3: Concatenate the 1024-channel local multi-scale feature map and the 256-channel global feature map along the channel dimension to obtain a 1280-channel aggregated feature map. Compress the number of channels to the original input channel number C using a 1×1 convolution. Finally, use residual connections to add the compressed aggregated features element-wise to the original input features of the module to obtain the final enhanced feature map. This process of enhancing the feature maps in three stages is repeated sequentially to obtain the first, second, and third enhanced feature maps.

[0046] S4: Decoder-side upsampling and cross-layer feature concatenation: The decoder adopts a hierarchical fusion architecture of step-by-step upsampling + cross-layer feature concatenation. Upsampling restores the spatial size of the feature map, while fusing multiple layers of enhanced features from the encoder to compensate for the shallow spatial details lost during downsampling, achieving accurate fusion of deep semantics and shallow details. The specific operation process is as follows: First, the third enhanced feature map (1 / 16 size of the original image, 2048 channels) is upsampled by bilinear interpolation by 2x, and then reduced to 256 channels by 1×1 convolution to obtain the first decoded feature map. Second, the second enhanced feature map is reduced to 256 channels, concatenated with the first decoded feature map, and then upsampled by 2x again after 3×3 convolution, batch normalization, ReLU activation to obtain the second decoded feature map. Finally, the first enhanced feature map is reduced to 256 channels, concatenated and fused with the second decoded feature map, and after convolution activation processing, the decoder's final-level fused feature map (1 / 4 size of the original image, 256 channels) is output.

[0047] S5: Edge Preserving Convolution Module Boundary Response Enhancement: The final-stage fused feature map from the decoder is input into the Edge Preserving Convolution (EPC) module. Parallel computation across two branches enhances building edge features and corrects boundaries, suppressing edge blurring and pixel sticking. The edge detection branch extracts horizontal and vertical gradient features using the Sobel operator. The Sobel operator employs a 3×3 standard convolution kernel, with the horizontal kernel set to [-1,0,1; -2,0,2; -1,0,1] and the vertical kernel to [-1,-2,-1; 0,0,0; 1,2,1]. For the input multi-channel feature map (256 channels), the horizontal gradient Gx and vertical gradient Gy are first calculated for each channel. Then, the average gradient along the channel dimension is taken to obtain the single-channel average gradient map. Finally, the gradient magnitude G = (Gx...) 2 +Gy 2 This is used as the edge response intensity map. This intensity map is normalized to the (0,1) interval using the Sigmoid activation function to generate the edge spatial attention weight map. The main segmentation branch extracts basic segmentation feature maps through two layers of 3×3 convolution, batch normalization, and ReLU activation.

[0048] Feature optimization is achieved through an adaptive edge enhancement algorithm, the core formula of which is as follows: In the formula: The basic segmentation features of the main branch are as follows. This is a learnable scaling parameter (initial value 1.0). This is the edge attention weight map. Finally, it is mapped by a 1×1 convolution to a 2-channel score feature map corresponding to the building and background categories.

[0049] S6: Background Noise Removal Processing of the Hybrid Attention Module: The final-stage fused feature map of the decoder is simultaneously input into the Hybrid Attention (HAM) module, which consists of a cascaded channel attention submodule and a spatial attention submodule. This module adaptively filters effective features and suppresses complex background noise such as roads, bare ground, and shadows. The channel attention submodule extracts global features of each channel through global max pooling and global average pooling, reconstructs them using a shared multilayer perceptron with a reduction ratio of 16, generates channel attention weights, and completes the weighted filtering of feature channels. The spatial attention submodule performs dimensional pooling, concatenation, and 7×7 convolution operations on the enhanced channel features to generate a spatial attention weight map, accurately suppressing invalid background region features and outputting a high-purity hybrid attention enhanced feature map.

[0050] The specific computational flow of the hybrid attention module is as follows: Channel attention submodule: Input features First, global max pooling and global average pooling are performed separately, resulting in two... dimensional vector , The two vector inputs are shared by a multilayer perceptron (MLP), which includes a dimensionality reduction layer (output dimension). ReLU activation, an upscaling layer (output dimension) ), respectively, to obtain the channel attention weight vectors , The two are added element-wise and then activated by a Sigmoid function to obtain the final channel attention weights. Channel enhancement features , ⊙ represents channel-by-channel multiplication.

[0051] Spatial Attention Submodule: Enhancing Features with Channels As input, perform max pooling and average pooling along the channel dimension respectively to obtain two results. The single-channel feature map is stitched together along the channel dimension to form 2 The tensor is reduced to 1 dimension by 7×7 convolution (padding=3). Then, a spatial attention weight map is generated through Sigmoid activation. Final output , ⊙ represents element-wise multiplication.

[0052] The channel attention submodule and the spatial attention submodule are executed in sequence, with channel attention executed first and spatial attention executed later, preserving the dimensionality of the output feature map.

[256] remains unchanged.

[0053] S7: Classification Score Fusion and Building Segmentation Mask Generation: The 256-channel enhanced feature map output from the HAM module is subjected to 1×1 convolution for dimensionality reduction, resulting in a 2-channel classification feature map. This map is then concatenated with the 2-channel edge enhancement score map output from the EPC module and fused using a 1×1 convolution to obtain the final 2-channel pixel classification score map. The score map is then upsampled by 8x bilinearly to restore it to the original image size of 512×512 pixels. The pixel scores are converted into a class probability distribution using the Softmax activation function. A threshold of 0.5 is used to classify pixels as building pixels; otherwise, they are classified as background pixels. This process ultimately generates a binarized building segmentation mask.

[0054] S8: Offline training strategy based on deep supervision and Focal-Dice hybrid loss: This step is the core process of offline iterative optimization of the model. By using multi-scale deep supervision and hybrid loss function, it solves the problems of imbalanced positive and negative samples in remote sensing images, missed detection of small buildings, and inaccurate boundary segmentation, thereby improving the model's generalization ability. It is specifically divided into 3 sub-steps.

[0055] Step S8.1: During model training, multi-path deep supervision is implemented on four layers of nodes, including the output features of the encoder in the third, fourth, and fifth stages and the final output features of the decoder. Specifically, the intermediate features of the three encoder layers are mapped to a 2-channel score map of the original image size through independent auxiliary classifiers (1×1 convolution + bilinear upsampling). The output features of the decoder are directly upsampled to match the original image size. The loss of the four features is calculated with the real label mask, realizing full-level constraints on features from shallow to deep layers.

[0056] Each auxiliary classifier has a consistent structure and does not share weights: for the encoder... Stage output feature map ( =3, 4, 5 correspond to 1 / 4, 1 / 8, and 1 / 16 sizes respectively), first, the number of channels is increased by a 1×1 convolution. Compress to 2 (corresponding to binary classification of buildings and background), then use bilinear interpolation upsampling to reduce the spatial size from... Enlarge to the original image size step by step (512×512). Upsampling factors were 4x, 8x, and 16x. The loss was calculated between the upsampled 2-channel score map and the ground truth label mask (512×512 binary map). The decoder's main output feature map (1 / 4 size, 256 channels) was also mapped to 2 channels by a 1×1 convolution, and then restored to 512×512 by 8x bilinear upsampling.

[0057] Step S8.2: Employ a hybrid loss function combining Focal loss and Dice loss to adapt to imbalanced remote sensing image samples. Focal loss sets the focusing parameters. Balance parameters This approach suppresses the gradient dominance of a large number of simple background samples and strengthens the learning weights of small-scale buildings and samples with difficult-to-distinguish edges. The Dice loss is calculated based on the intersection and union relationship between the predicted and real regions, optimizing the overall segmentation overlap. The total loss for a single path is the direct sum of the two types of losses. The formula for calculating the total loss for multi-path deep supervision is as follows: In the formula: This is the decoder's main output loss (weight 1.0). The three auxiliary losses of the encoder are represented by a weight of 0.4 for each.

[0058] Step S8.3: Calculate the gradients of the network weights at each layer using the backpropagation algorithm based on the total loss value. Dynamically update the model parameters using the AdamW optimizer and cosine annealing learning rate. Iterate training until the validation set loss value stabilizes and the accuracy metric no longer improves, at which point the model is considered converged. Persistently store the converged optimal network weight parameters on a storage server for subsequent online inference.

[0059] S9: Image Patch Stitching and Result Output: After the GPU completes single image patch inference, the binary segmentation mask is sent back to the CPU memory via the PCIe bus. The CPU retrieves the coordinate information of each image patch recorded during the sliding window segmentation stage and fills it into the full-image mask matrix patch by patch. To address the pixel conflict problem in overlapping areas of image patches, a pixel mean fusion strategy is adopted: the arithmetic mean of all predicted pixel values ​​at the same location in the overlapping area is taken; if the mean is ≥0.5, it is determined to be a building; otherwise, it is determined to be background, thus completely eliminating stitching boundary artifacts. Finally, a complete segmentation mask that perfectly matches the size of the original remote sensing image is generated. The mask is superimposed on the original image and pushed to the terminal device for visualization rendering, while being archived and stored in GeoTIFF format.

[0060] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for extracting buildings from remote sensing images based on the fusion of global and local features, based on a computer device including a data receiving server, a computing server, and a storage server, wherein the computing server is equipped with a graphics processor, characterized in that, Includes the following steps: S1: The data receiving server receives the raw remote sensing image data stream through a satellite communication link, demodulates, decodes, and restores the format to generate the raw remote sensing image, and transmits it to the central processing unit of the computing server cluster via Ethernet; the central processing unit sequentially performs radiometric calibration, atmospheric correction, band registration, and fusion on the raw remote sensing image to generate a multi-channel fused image, and performs sliding window segmentation on the multi-channel fused image to generate several image blocks, performs normalization processing on each image block, and then moves it to the graphics processing unit's video memory of the computing server cluster; S2: The graphics processor inputs the normalized image patches into the encoder backbone network. The encoder backbone network adopts the ResNet50 architecture and extracts and outputs at least three intermediate feature maps of different spatial scales through multi-stage convolution and downsampling operations. S3: The graphics processor performs multi-scale long-range aggregation processing on each intermediate feature map. The multi-scale long-range aggregation processing includes: The first branch processing: After channel dimensionality reduction of the input feature map, multiple dilated convolutions with different dilation rates are executed in parallel. The outputs of each dilated convolution are concatenated along the channel dimension to obtain a local multi-scale context feature map. The second branch processing: After channel dimensionality reduction of the input feature map, the feature map is flattened into a sequence matrix, linear self-attention operation is performed, global long-range dependencies between pixels are established, and a global context feature map is obtained. The local multi-scale context feature map is fused with the global context feature map and compressed to the number of channels of the input feature map. The residual is then superimposed with the input feature map to output a multi-scale enhanced feature map. S4. The graphics processor inputs the multi-scale enhanced feature maps output in step 3 into the decoder, performs upsampling operations in sequence, and performs cross-layer concatenation and convolution fusion of the upsampled feature maps with the corresponding multi-scale enhanced feature maps, and outputs the final-level fused feature map of the decoder. S5: The graphics processor performs edge-preserving convolution processing on the final-level fusion feature map of the decoder, including: extracting the edge response intensity map of the final-level fusion feature map of the decoder and converting it into an edge space attention weight map, while extracting the main segmentation feature map, and performing element-wise weighted fusion of the main segmentation feature map and the edge space attention weight map to output the edge enhancement feature map. S6: The graphics processor performs hybrid attention processing on the final-stage fused feature map of the decoder, including: sequentially performing channel attention weighting and spatial attention weighting, filtering out background noise response, and outputting attention-enhanced feature map; S7: The graphics processor performs channel fusion of the edge enhancement feature map and the attention enhancement feature map, and then performs convolution mapping to obtain the pixel classification score map. After upsampling the pixel classification score map to the original size of the image block, it is segmented by activation function and threshold to generate building segmentation mask. S8: The graphics processor sends the building segmentation mask back to the central processing unit (CPU). The CPU stitches the segmentation masks of each image block according to the sliding window segmentation position to generate a complete segmentation mask that matches the size of the original remote sensing image, and outputs it to the terminal display device and storage server.

2. A method for extracting buildings from remote sensing images based on the fusion of global and local features according to claim 1, characterized in that, The band registration and fusion described in step S1 specifically includes: using the phase correlation method to detect the offset between the panchromatic band and the multispectral band and performing registration; using the Gram-Schmidt fusion algorithm or the principal component analysis fusion algorithm to inject panchromatic band details into the multispectral band to generate the multichannel fused image.

3. A method for extracting buildings from remote sensing images based on the fusion of global and local features according to claim 1, characterized in that, The sliding window segmentation in step S1 uses a fixed-size sliding window, and overlapping areas are set between adjacent image blocks in both the row and column directions.

4. A method for extracting buildings from remote sensing images based on the fusion of global and local features according to claim 1, characterized in that, In step S3, the channel dimensionality reduction of the first branch compresses the number of input feature map channels to a first preset number of channels. The dilation rates of the multiple dilated convolutions with different dilation rates are 1, 6, 12, and 18, respectively.

5. A method for extracting buildings from remote sensing images based on the fusion of global and local features according to claim 1, characterized in that, In step S3, the channel dimensionality reduction of the second branch compresses the number of input feature map channels to a second preset number. The linear self-attention operation uses a kernel function. By performing feature mapping on the query matrix and key matrix, the computational complexity of self-attention is reduced to the linear order of the sequence length.

6. A method for extracting buildings from remote sensing images based on the fusion of global and local features according to claim 1, characterized in that, The extraction of the edge response intensity map in step S5 specifically includes: using the Sobel operator to calculate the gradient response in the horizontal direction and the gradient response in the vertical direction respectively, superimposing the gradient responses in the two directions and normalizing them by the S-shaped activation function to generate the edge spatial attention weight map.

7. A method for extracting buildings from remote sensing images based on the fusion of global and local features according to claim 1, characterized in that, The element-wise weighted fusion described in step S5 uses the formula ,in Main segmentation feature map For the attention weight map in the edge space, For learnable scaling parameters.

8. A method for extracting buildings from remote sensing images based on the fusion of global and local features according to claim 1, characterized in that, The channel attention weighting in step S6 includes: performing global max pooling and global average pooling on the input feature map respectively, adding them after mapping through a shared multilayer perceptron and activating them to generate channel attention weights, and multiplying them element-wise with the input feature map; the spatial attention weighting includes: performing global max pooling and global average pooling on the channel-weighted feature map respectively, concatenating them and generating spatial attention weights through convolution and activation, and multiplying them element-wise with the channel-weighted feature map.

9. A method for extracting buildings from remote sensing images based on the fusion of global and local features according to claim 1, characterized in that, The training steps in step S2 are as follows: Auxiliary classifiers are set at the output of at least three intermediate stages of the encoder and the final output of the decoder. The output of each auxiliary classifier is upsampled and the loss is calculated with the real label. The loss of each layer is weighted and summed to obtain the total loss. The network weights are updated based on the total loss. The loss is a hybrid loss of Focal loss and Dice loss.

10. A method for extracting buildings from remote sensing images based on the fusion of global and local features according to claim 1, characterized in that, The data receiving server and the computing server cluster transmit data via 10 Gigabit Ethernet and TCP / IP protocol, or via 5G mobile communication network and UDP protocol.

Images