A method for extracting buildings in high-altitude cold regions based on Swing Transformer and feature fusion

Abstract

The application discloses a kind of high-cold mountainous area building extraction method based on Swin Transformer and feature fusion, which comprises the following steps: obtaining high-resolution three-band remote sensing image, constructing data label with an error range of two pixels;Image cropping is carried out on the source image and label, and data enhancement is carried out by rotation and mirroring;In view of the problem that buildings in the image are heavily shadowed, the context information around the building is processed by fusing the difference features of color space and normalized color space, the color space brightness feature, the clustering feature and the vegetation shadow feature;Divide the training set and the validation set, and construct the high-precision high-cold mountainous area building image dataset;Replace the model loss function with a combination of binary cross-entropy loss function and square die loss function to guide the training, effectively improving the feature extraction capability of the model.

Description

Technical Field

[0001] This invention belongs to the field of remote sensing image processing technology, specifically relating to a method, equipment, and medium for extracting buildings in high-altitude and cold mountainous areas based on Swing Transformer and feature fusion. Background Technology

[0002] High-altitude and cold mountainous areas are generally characterized by high altitude, low temperatures, fragmented terrain, and numerous irregular buildings in rural areas. The climate in these areas makes traditional manual surveying and construction extremely costly. However, with the development of border regions, the demand for infrastructure such as transportation and energy is surging. At the same time, building extraction can help optimize site selection and assess disaster risks, which urgently requires the rapid and accurate acquisition of building information.

[0003] Low-altitude remote sensing technology, primarily using UAV remote sensing, and high-resolution satellite remote sensing enable scientific research and production work to obtain massive amounts of high-resolution remote sensing image data. This allows for the efficient acquisition of surface information, the display of rich details of ground features, and the reflection of relationships between ground features. It is widely used in fields such as ground feature detection, surveying and mapping, and land use, and can provide effective data support for buildings in high-altitude and cold mountainous areas.

[0004] These data require digital processing techniques to extract target information. Ground feature extraction from high-resolution remote sensing imagery has gone through three stages: The first stage is manual extraction based on prior knowledge. This involves identifying the relationship between ground features and their surrounding environment using prior knowledge, requiring manual visual interpretation and analysis of building information in the image. This method is highly accurate, but costly and labor-intensive, making it unsuitable for efficiently acquiring information from large datasets over long periods. The second stage, with increasingly complex surface objects, saw research into automated building extraction. Automated extraction relies on building features such as geometric, spatial semantic, and spectral characteristics. By establishing relationships between feature data, a model for automated building extraction is constructed. However, automated extraction requires pre-setting complex rules and algorithms, increasing workload and resulting in low efficiency and relatively poor accuracy. The third stage is automated building extraction based on deep learning. The rapid development of deep learning has driven the development of algorithms in various related fields. Its powerful ability to automatically learn and extract features from data without requiring complex pre-setting algorithms, coupled with high efficiency and accuracy, has made it a leading force in image processing. Among numerous algorithms, CNN-based algorithms suffer from limitations in local information perception due to convolution kernel constraints, which still affects the final feature representation. SwinTransformer, through its hierarchical window mechanism, achieves efficient global modeling and is more competitive in this field.

[0005] Building extraction in high-altitude and cold mountainous areas presents even greater challenges. Buildings in these regions often have dense shadows, and the complex background of the plateau and mountains includes varying lighting conditions, diverse vegetation, and complex soil structures. Furthermore, the chaotic arrangement of buildings and their complex and variable scale and geometric features make it difficult for general algorithms to simultaneously achieve accuracy in extracting multi-scale targets and maintain the geometric integrity of buildings. At the same time, deep learning algorithms generally suffer from poor sensitivity to external data sources, and their application in building extraction in high-altitude and cold regions is still in its early stages. Therefore, further research is needed to achieve more accurate extraction under complex feature conditions. Summary of the Invention

[0006] To address the issue of insufficient extraction completeness when extracting irregularly shaped buildings in complex environments of high-altitude and cold mountainous areas, this invention provides a method for extracting buildings in high-altitude and cold mountainous areas based on Swing Transformer and feature fusion. This method effectively improves the shape completeness, edge completeness, and accuracy of building extraction in high-altitude and cold mountainous areas by optimizing the Swing Transformer model.

[0007] The present invention provides a method for extracting buildings in high-altitude and cold mountainous areas based on Swing Transformer and feature fusion, characterized by the following steps:

[0008] Step 1: Acquire high-precision remote sensing image data and perform data annotation, constructing data labels with an error range of two pixels;

[0009] Step 2: Crop the image data from Step 1 and perform data enhancement through image rotation and mirror flipping;

[0010] Step 3: Apply a multi-feature fusion image shadow processing method to the image data after cropping and enhancement in Step 2. Integrate multiple features to identify shadow areas. Process only the shadow areas without changing the features of non-shadow areas to reduce shadow intensity and mitigate the impact of shadows on the feature information of the area.

[0011] Step 4: Divide the data processed in Step 3 into data segments;

[0012] Step 5: Construct the encoder. Add a cross-dimensional feature fusion calculation module to the Swin Transformer architecture to extract features from the image data segmented in Step 4. Input the image dataset segmented in Step 4, divide each image into fixed-size data blocks, and map the pixel values ​​of these data blocks to a high-dimensional space using a linear embedding module to output a feature map. Perform cross-dimensional attention feature fusion calculation on the output feature map, including a convolutional attention module and a 1×1 convolution operation. Given a feature map F, the calculation method is as follows:

[0013]

[0014] CAM(F)=σ(MLP(AvgPool(F))+MLP(MaxPool(F)))

[0015] SAM(F)=σ(conv 7×7 ([AvgPool(F)];MaxPool(F)]))

[0016] Where F' is the feature output after computation, SAM represents spatial attention computation, and CAM represents channel attention computation. conv represents the multiplication operation, σ represents the sigmoid function, AvgPool represents average pooling, MaxPool represents max pooling, and conv represents multiplication. 7×7 This indicates a convolution operation with a 7×7 kernel;

[0017] Finally, a 1×1 convolution operation is added after the convolutional attention module and introduced before the LN layer of the linear embedding module, i.e., the Layer Normalization layer.

[0018] The input is then fed into a combination module consisting of four-stage slice merging modules and different numbers of Swing Transformer algorithm blocks. Each stage constructs a hierarchical feature map, and the generated feature maps are f1, f2, f3 and f4 in sequence, realizing hierarchical feature extraction, which helps the model process high-resolution images. Among them, the cascaded window multi-head self-attention module (W-MSA) and the shifted window multi-head self-attention module (SW-MSA) in the Swing Transformer algorithm block are used to extract local and global features of the image.

[0019] Step 6: Construct a decoder to fuse multi-scale features. The features encoded by the encoder are used by a decoder that fuses feature maps of multiple resolutions to output the building extraction results.

[0020] Step 7: Replace the loss function with a combined loss function L of binary cross-entropy loss and squared dice loss. BCE-SDice Training is performed, and during the model training process, the model is guided to update its parameters through optimization algorithms, thereby continuously improving the accuracy of predictions.

[0021] Step three, which involves using a multi-feature fusion method to process the image shadows of the source image, further includes:

[0022] (3) Obtain the difference features f1 between the original color space and the normalized color space, the color space brightness features f2, the spectral characteristic clustering features f3, and the vegetation shadow features f4 on the image pixel matrix, where:

[0023]

[0024] f2=αR+βG+γB

[0025] f3 is a matrix obtained by clustering pixels according to their spectral characteristics using K-means to identify shadow regions.

[0026] f4 = G - min(R, B)

[0027] R, G, and B represent the values ​​of the R, G, and B channels of the image in the RGB color space, respectively. α, β, and γ represent custom adjustment coefficients, and α + β + γ = 1.

[0028] (4) All features are merged to form an overall feature F, mathematically expressed as F = a × f1 + b × f2 + c × f3 + d × f4, where a, b, c, and d are feature weights. The merged feature is used to determine whether a region is a shaded area. The determination method is as follows:

[0029]

[0030] When the feature value F is greater than the threshold V, it is a shadow; otherwise, it is not a shadow.

[0031] (3) When the above is a shadow area, the shadow area is converted to the HSV (Hue Saturation Value) color space, the value of the luminance V channel is increased to improve the brightness of the shadow area, and the context information of the shadow part is restored without changing the image features of other areas.

[0032] Furthermore, the specific scheme for constructing the decoder in step six, which integrates multi-scale features, is as follows:

[0033] The encoder output feature map is input into the pyramid pooling module. A 1×1 convolution is applied to adjust the channel dimensions. Four-level pyramid pooling is used, employing 1×1, 2×2, 3×3, and 6×6 windows respectively to pool the input feature map f4. After upsampling, the original feature maps are concatenated to form a new feature map P1. P1 is then upsampled three times to generate three scale feature maps P2, P3, and P4. Features at each scale are integrated through a cross-scale feature fusion mechanism and skip connections. The feature maps f4 from the last three stages of the encoder are then sequentially processed. i With feature map P i The feature map F is generated by merging and resizing. i =f i ⊕P iThen, F1, F2, F3, and F4 are used as global multi-scale features and connected and fused in the channel dimension to combine low-resolution high-level features with high-resolution low-level features; finally, the mask result of building extraction is output through convolution operation.

[0034] Furthermore, step seven involves replacing the default loss function with L. BCE-SDice The training is performed using a combination of loss functions, specifically as follows:

[0035] Combining the binary cross-entropy loss and the squared dice loss, the mathematical expression is:

[0036] L BCE-SDice =(1-α)L BCE +αL SquareDice

[0037]

[0038] Among them, L BCE-SDice L represents the portfolio loss. BCE L represents the binary cross-entropy loss. SquaredDice y represents the loss of a square die. i ∈{0,1} is the binary label of pixel i, where 1 represents the positive class and 0 represents the negative class. Let be the probability that pixel i is predicted to be of the positive class. α represents the probability that pixel i is predicted to be a negative class, N represents the total number of pixels, smooth is the smoothing parameter, and α∈[0,1] is the set parameter.

[0039] A second aspect of the present invention provides a building extraction device, comprising a memory and a controller connected in sequence, wherein the memory stores a computer program, and the controller is used to read the computer program and execute the building extraction method for high-altitude cold mountainous areas based on Swing Transformer and feature fusion as described in the first aspect.

[0040] A third aspect of the present invention provides a computer-readable storage medium storing instructions. When the instructions are executed on a computer, the controller is configured to read the computer program and execute the method for extracting buildings in high-altitude and cold mountainous areas based on Swing Transformer and feature fusion as described in the first aspect.

[0041] Compared with the prior art, the present invention has at least the following advantages and beneficial effects:

[0042] This invention enhances the model's feature extraction capabilities by optimizing and improving image processing, feature fusion extraction, loss function optimization, and decoder. It achieves clearer, more accurate, and more complete building outlines for various types of buildings, realizing automatic extraction of buildings from remote sensing images and effectively improving the accuracy and shape integrity of the extraction. Attached Figure Description

[0043] Figure 1 This is a flowchart of the method of the present invention.

[0044] Figure 2 This is a schematic diagram of an encoder based on the Swing Transformer.

[0045] Figure 3 This is a schematic diagram of the structure of the convolutional attention module.

[0046] Figure 4 This is a schematic diagram of the pyramid pooling module (PPM) structure.

[0047] Figure 5 This is a schematic diagram of the decoder structure. Detailed Implementation

[0048] This invention provides a technical solution: a method for extracting buildings in high-altitude cold mountainous areas based on Swing Transformer and feature fusion, characterized by the following steps:

[0049] Step 1: Acquire remote sensing image data of a high-altitude mountainous area with a resolution of 0.05m and perform data annotation, constructing data labels with an error range of two pixels;

[0050] Step 2: The image data from Step 1 is cropped and augmented using image rotation and mirror flipping. The cropped size is 512 pixels × 512 pixels. The image data is then augmented using image rotation and mirror flipping. The rotation angles involve clockwise rotation of 90°, 180°, and 270°, and the mirror flipping involves horizontal and vertical mirror flipping. Finally, 30,810 image data samples are obtained.

[0051] Step 3: Apply a multi-feature fusion image shadow processing method to the image data after cropping and enhancement in Step 2. This method integrates multiple features to identify shadow regions and processes only the shadow regions without altering the features of non-shadow regions. This reduces the shadow intensity and mitigates the impact of shadows on the feature information of the area in which they occur. The specific processing method is as follows:

[0052] (1) Obtain the difference features f1 between the original color space and the normalized color space, the color space brightness features f2, the spectral characteristic clustering features f3, and the vegetation shadow features f4 on the image pixel matrix, where:

[0053]

[0054] f2=αR+βG+γB

[0055] f3 is a matrix obtained by clustering pixels according to their spectral characteristics using K-means to identify shadow regions.

[0056] f4 = G - min(R, B)

[0057] R, G, and B represent the values ​​of the R, G, and B channels of the image in the RGB color space, respectively. α, β, and γ represent custom adjustment coefficients, and α + β + γ = 1.

[0058] (2) All features are merged to form an overall feature F, mathematically expressed as F = a × f1 + b × f2 + c × f3 + d × f4, where a, b, c, and d are feature weights. The merged feature is used to determine whether a region is a shaded area. The determination method is as follows:

[0059]

[0060] When the feature value F is greater than the threshold V, it is a shadow; otherwise, it is not a shadow.

[0061] (3) When the above is a shadow area, the shadow area is converted to the Hue Saturation Value (HSV) color space, the value of the brightness V channel is increased to improve the brightness of the shadow area, and the context information of the shadow part is restored without changing the image features of other areas.

[0062] Step 4: Divide the data processed in Step 3 into training set, validation set and test set;

[0063] Step 5: Construct the encoder. Add a cross-dimensional feature fusion calculation module to the Swin Transformer architecture to extract features from the image data segmented in Step 4. Input the image dataset segmented in Step 4, divide each image into fixed-size data blocks, and map the pixel values ​​of these data blocks to a high-dimensional space using a linear embedding module to output a feature map. Perform cross-dimensional attention feature fusion calculation on the output feature map, including a convolutional attention module and a 1×1 convolution operation. Given a feature map F, the calculation method is as follows:

[0064]

[0065] CAM(F)=σ(MLP(AvgPool(F))+MLP(MaxPool(F)))

[0066] SAM(F)=σ(conv7×7 ([AvgPool(F)];MaxPool(F)]))

[0067] Where F' is the feature output after computation, SAM represents spatial attention computation, and CAM represents channel attention computation. conv represents the multiplication operation, σ represents the sigmoid function, AvgPool represents average pooling, MaxPool represents max pooling, and conv represents multiplication. 7×7 This indicates a convolution operation with a 7×7 kernel;

[0068] Finally, a 1×1 convolution operation is added after the convolutional attention module and introduced before the LayerNormalization (LN) layer of the linear embedding module;

[0069] The input is then fed into a combination module consisting of four-stage slice merging modules and different numbers of Swing Transformer algorithm blocks. Each stage constructs a hierarchical feature map, and the generated feature maps are f1, f2, f3 and f4 in sequence, realizing hierarchical feature extraction, which helps the model process high-resolution images. Among them, the cascaded window multi-head self-attention module (W-MSA) and the shifted window multi-head self-attention module (SW-MSA) in the Swing Transformer algorithm block are used to extract local and global features of the image.

[0070] Step 6: Construct a decoder to fuse multi-scale features. The features encoded by the encoder are then processed by a decoder that fuses feature maps of multiple resolutions to output the building extraction results.

[0071] The encoder output feature map is input into the pyramid pooling module. A 1×1 convolution is applied to adjust the channel dimensions. Four-level pyramid pooling is used, employing 1×1, 2×2, 3×3, and 6×6 windows respectively to pool the input feature map f4. After upsampling, the original feature maps are concatenated to form a new feature map P1. P1 is then upsampled three times to generate three scale feature maps P2, P3, and P4. Features at each scale are integrated through a cross-scale feature fusion mechanism and skip connections. The feature maps f4 from the last three stages of the encoder are then sequentially processed. i With feature map P i The data is then fused and resized to generate feature maps. Then, F1, F2, F3, and F4 are used as global multi-scale features and connected and fused in the channel dimension to combine low-resolution high-level features with high-resolution low-level features; finally, the mask result of building extraction is output through convolution operation.

[0072] Step 7: Replace the loss function with a combined loss function L of binary cross-entropy loss and squared dice loss. BCE-SDice Training is conducted, and during the model training process, the model is guided to update its parameters through optimization algorithms, thereby continuously improving the accuracy of predictions.

[0073] Replace the default loss function with L BCE-SDice The training is performed using a combination of loss functions, specifically as follows:

[0074] Combining the binary cross-entropy loss and the squared dice loss, the mathematical expression is:

[0075] L BCE-SDice =(1-α)L BCE +αL SquareDice

[0076]

[0077] Among them, L BCE-SDice L represents the portfolio loss. BCE L represents the binary cross-entropy loss. SquaredDice y represents the loss of a square die. i ∈{0,1} is the binary label of pixel i, where 1 represents the positive class and 0 represents the negative class. Let be the probability that pixel i is predicted to be of the positive class. Let be the probability that the prediction result of pixel i is negative, N represents the total number of pixels, smooth is the smoothing parameter, and α∈[0,1] is the set parameter. When calculating, α=0.7.

Claims

1. A method for extracting buildings in high-altitude cold mountainous areas based on Swing Transformer and feature fusion, characterized in that... The extraction method includes the following steps: Step 1: Acquire high-precision remote sensing image data and perform data annotation, constructing data labels with an error range of two pixels; Step 2: Crop the image data from Step 1 and perform data enhancement through image rotation and mirror flipping; Step 3: Apply a multi-feature fusion image shadow processing method to the image data after cropping and enhancement in Step 2. Integrate multiple features to identify shadow areas. Process only the shadow areas without changing the features of non-shadow areas to reduce shadow intensity and mitigate the impact of shadows on the feature information of the area. Step 4: Divide the data processed in Step 3 into data segments; Step 5: Construct the encoder. Add a cross-dimensional feature fusion calculation module to the Swin Transformer architecture to extract features from the image data segmented in Step 4. Input the image dataset segmented in Step 4, divide each image into fixed-size data blocks, and map the pixel values ​​of these data blocks to a high-dimensional space using a linear embedding module to output a feature map. Perform cross-dimensional attention feature fusion calculation on the output feature map, including a convolutional attention module and a 1×1 convolution operation. Given a feature map F, the calculation method is as follows: CAM(F)=σ(MLP(AvgPool(F))+MLP(MaxPool(F))) SAM(F)<σ(conv) 7×7 ([AvgPool(F)]; MaxPool(F)]) Where F' is the feature output after computation, SAM represents spatial attention computation, and CAM represents channel attention computation. conv represents the multiplication operation, σ represents the sigmoid function, AvgPool represents average pooling, MaxPool represents max pooling, and conv represents multiplication. 7 ×7 This indicates a convolution operation with a 7×7 kernel; Finally, a 1×1 convolution operation is added after the convolutional attention module and introduced before the LN layer of the linear embedding module, i.e., the Layer Normalization layer. The input is then fed into a combination module consisting of four-stage slice merging modules and different numbers of Swing Transformer algorithm blocks. Each stage constructs a hierarchical feature map, and the generated feature maps are f1, f2, f3 and f4 in sequence, realizing hierarchical feature extraction, which helps the model process high-resolution images. Among them, the cascaded window multi-head self-attention module (W-MSA) and the shifted window multi-head self-attention module (SW-MSA) in the Swing Transformer algorithm block are used to extract local and global features of the image. Step 6: Construct a decoder to fuse multi-scale features. The features encoded by the encoder are used by a decoder that fuses feature maps of multiple resolutions to output the building extraction results. Step 7: Replace the loss function with a combined loss function L of binary cross-entropy loss and squared dice loss. BCE-SDice Training is performed, and during the model training process, the model is guided to update its parameters through optimization algorithms, thereby continuously improving the accuracy of predictions.

2. The method for extracting buildings in high-altitude cold mountainous areas based on Swing Transformer and feature fusion as described in claim 1, characterized in that... In step three, the image shadow processing method using multi-feature fusion is applied to the source image, which further includes: (1) obtaining the difference features f1 between the original color space and the normalized color space, the color space brightness features f2, the spectral characteristic clustering features f3, and the vegetation shadow features f4 on the image pixel matrix, wherein: f2=αR+βG+γB f3 is a matrix obtained by clustering pixels according to their spectral characteristics using K-means to identify shadow regions. f4 = G - min(R, B) R, G, and B represent the values ​​of the R, G, and B channels of the image in the RGB color space, respectively. α, β, and γ represent custom adjustment coefficients, and α + β + γ = 1. (2) All features are merged to form an overall feature F, mathematically expressed as F = a × f1 + b × f2 + c × f3 + d × f4, where a, b, c, and d are feature weights. The merged feature is used to determine whether a region is a shaded area. The determination method is as follows: When the feature value F is greater than the threshold V, it is a shadow; otherwise, it is not a shadow. (3) When the above is a shadow area, the shadow area is converted to the HSV (Hue Saturation Value) color space, the value of the luminance V channel is increased to improve the brightness of the shadow area, and the context information of the shadow part is restored without changing the image features of other areas.

3. The method for extracting buildings in high-altitude and cold mountainous areas based on Swing Transformer and feature fusion as described in claim 1, characterized in that... The specific scheme for constructing the decoder and fusing multi-scale features in step six is ​​as follows: The encoder output feature map is input into the pyramid pooling module. A 1×1 convolution is applied to adjust the channel dimensions. Four-level pyramid pooling is used, employing 1×1, 2×2, 3×3, and 6×6 windows respectively to pool the input feature map f4. After upsampling, the original feature maps are concatenated to form a new feature map P1. P1 is then upsampled three times to generate three scale feature maps P2, P3, and P4. Features at each scale are integrated through a cross-scale feature fusion mechanism and skip connections. The feature maps f4 from the last three stages of the encoder are then sequentially processed. i With feature map P i The feature map F is generated by merging and resizing. i =f i ⊕P i Then, F1, F2, F3, and F4 are used as global multi-scale features and connected and fused in the channel dimension to combine low-resolution high-level features with high-resolution low-level features; finally, the mask result of building extraction is output through convolution operation.

4. The method for extracting buildings in high-altitude cold mountainous areas based on Swing Transformer and feature fusion as described in claim 1, characterized in that... In step seven, the default loss function is replaced with L. BCE-SDice The training is performed using a combination of loss functions, specifically as follows: Combining the binary cross-entropy loss and the squared dice loss, the mathematical expression is: L BCE-SDice =(1-α)L BCE +αL SquareDice Among them, L BCE-SDice L represents the portfolio loss. BCE L represents the binary cross-entropy loss. SquaredDice y represents the loss of a square die. i ∈{0,1} is the binary label of pixel i, where 1 represents the positive class and 0 represents the negative class. Let be the probability that pixel i is predicted to be of the positive class. α represents the probability that pixel i is predicted to be a negative class, N represents the total number of pixels, smooth is the smoothing parameter, and α∈[0,1] is the set parameter.

Images