A method and apparatus for extracting buildings from remote sensing images based on a dual-path skip-attention mechanism.
Patent Information
- Application Number
- CN202311419143.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-10-30
- Publication Date
- 2026-09-01
- Estimated Expiration
- 2043-10-30
AI Technical Summary
当前的遥感图像的建筑物提取方法主要基于U型编码解码架构，主要集中于采用不同特征提取模块和细化边缘处理上，但基于遥感图像高密度的信息事实，对于下采样过程中丢失的信息恢复和感受野大小权衡建筑物和背景上仍然存在问题和挑战
[0023]1、本发明基于U型编码解码架构，具备一定的普适性，适用于绝大部分可见光遥感图像数据集的建筑物自动化提取。
Smart Images
Figure CN117475307B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of image processing technology, and more specifically, relates to a method for extracting buildings from remote sensing images based on a dual-path skip attention mechanism. Background Technology
[0002] Building extraction is an important subtask in remote sensing image segmentation, typically used to automate the rendering of remote sensing images. Current methods for building extraction from remote sensing images are mainly based on a U-shaped encoder-decoder architecture, focusing primarily on employing different feature extraction modules and refining edge processing. However, given the high-density information in remote sensing images, problems and challenges remain regarding the recovery of information lost during downsampling and the balance between building and background in terms of receptive field size. Summary of the Invention
[0003] The present invention aims to overcome the above-mentioned shortcomings of the prior art and provides a method and apparatus for extracting buildings from remote sensing images based on a dual-path skip attention mechanism.
[0004] This invention, based on the existing U-shaped encoding and decoding architecture, incorporates a dual-path skip-attention mechanism and a global-local focusing module. This highlights salient building features while capturing both global dependencies and local corner areas, mitigating the problem of missed or false detections in local regions within information-dense remote sensing images. The dual-path skip-attention mechanism integrates more significant information features from the encoding process and suppresses background information. Combined with the global-local focusing module, it effectively improves the accuracy of building extraction from remote sensing images.
[0005] To achieve the above objectives, according to one aspect of the present invention, a method for extracting buildings from remote sensing images based on a dual-path skip attention mechanism includes:
[0006] Collect remote sensing datasets and corresponding mask label data, and train the constructed DSAUNet model using the datasets to obtain the trained DSAUNet model;
[0007] Input remote sensing image data to be processed, and analyze and calculate it through the trained DSAUNet model to obtain building mask images.
[0008] In some alternative implementations, the collection of remote sensing datasets and corresponding mask label data includes:
[0009] The collected remote sensing images and mask label data were cropped to a size of 512×512. Data augmentation was then performed to expand the dataset, including horizontal and vertical flipping, rotation, and blurring. The training, validation, and test sets were randomly divided in a 3:1:1 ratio.
[0010] In some alternative implementations, the constructed DSAUNet model includes:
[0011] Based on the U-shaped encoder-decoder architecture, a dual-path skip attention mechanism is used to replace the skip connection of the fourth layer, the first three layers use an attention gating mechanism, and a global-local focusing module is integrated before the decoder.
[0012] In some alternative implementations, the dual-path skip attention mechanism includes:
[0013] The output of the fourth coding layer is processed using 3×3 convolution, batch normalization, and ReLU activation function, and then passed through an attention gating mechanism. Finally, it is concatenated with the upsampled result of the decoding layer.
[0014] In some alternative implementations, the global-local focusing module includes:
[0015] The input features are first processed through smoothing blocks, then through global and local focusing modules, and finally output after channel concatenation. In the global focusing module, a multi-scale structure is employed, using three pairs of depthwise stripe convolutions, with kernel sizes of 7, 11, and 21 for each branch, followed by 1×1 convolutions. In the local focusing module, an encoder using an intrinsic dictionary is employed, inputting the encoded features into a codebook and using a set of scaling factors to map the corresponding location information. Then, fully connected layers are used, activated by a sigmoid function. Finally, the outputs of the smoothing blocks are combined for channel multiplication and addition.
[0016] In some alternative implementations, the trained DSAUNet model includes:
[0017] Input the training data into the DSAUNet model, set the optimizer to AdamW, and the initial learning rate to 1e-4. Calculate the loss and update the network parameters using BCELoss supervision; select the best training weights from the validation set and save the weight file.
[0018] In some alternative implementations, the analysis and computation performed using the trained DSAUNet model includes:
[0019] Load the saved weight file, extract buildings from the test set images, obtain the final building extraction mask, and perform qualitative and quantitative analysis.
[0020] A second aspect of the invention relates to a computer-readable storage medium having a program stored thereon that, when executed by a processor, implements the remote sensing image building extraction method based on a dual-path skip attention mechanism of the present invention.
[0021] A third aspect of the present invention relates to a computing device including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, it implements the remote sensing image building extraction method based on a dual-path skip attention mechanism of the present invention.
[0022] The advantages of this invention are as follows:
[0023] 1. This invention is based on a U-shaped encoding and decoding architecture, which has a certain degree of universality and is applicable to the automatic extraction of buildings from most visible light remote sensing image datasets.
[0024] 2. This invention uses a dual-path skip attention mechanism instead of skip connections, which can effectively balance building information and background information and alleviate the problem of feature information loss during downsampling.
[0025] 3. This invention combines a global-local focusing module, which can effectively focus on local corner buildings and improve the building extraction accuracy. Attached Figure Description
[0026] Figure 1 This is a flowchart of the remote sensing image building extraction method based on the dual-path skip attention mechanism of the present invention.
[0027] Figure 2 This is a model diagram of the remote sensing image building extraction method based on the dual-path skip attention mechanism of the present invention.
[0028] Figure 3 This is a diagram of the dual-path skipping attention mechanism of the present invention.
[0029] Figure 4 This is a diagram of the global-local focusing module of the present invention.
[0030] Figure 5 The image shows the result of the remote sensing image building extraction method based on the dual-path skip attention mechanism of this invention. Detailed Implementation
[0031] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.
[0032] Example 1
[0033] The method for extracting buildings from remote sensing images based on a dual-path skip attention mechanism of the present invention is implemented as follows: Figure 1 As shown, it mainly includes the following steps:
[0034] Step 1: Collect remote sensing datasets and corresponding mask label data, and train the constructed DSAUNet model using the datasets to obtain the trained DSAUNet model.
[0035] Step 2: Input the remote sensing image data to be processed, and analyze and calculate it through the trained DSAUNet model to obtain the building mask image.
[0036] Figure 2 This is a model diagram of a remote sensing image building extraction method based on a dual-path skip attention mechanism. Specifically, it can be divided into an encoding process and a decoding process:
[0037] Encoding process:
[0038] The acquired building remote sensing dataset images were processed through five coding layers, i.e., four max pooling downsampling operations, resulting in a total downsampling of the image size by 16 times.
[0039] Each coding layer consists of two 3×3 convolutions, batch normalization, and ReLU activation.
[0040] Decoding process:
[0041] The decoding process gradually restores the image size and generates the corresponding segmentation mask by upsampling. Specifically:
[0042] The deep feature output of the fifth layer of the encoder first passes through the global-local focusing module to establish global and local relationships between deep features, thereby enhancing the ability to capture feature centers.
[0043] For the fourth layer decoder, the output of the encoder's fourth layer, along with its initial processing (3×3 convolution, batch normalization, and ReLU activation) and upsampling result, is input into the dual-path skip attention mechanism module. This module focuses on finer-grained edge features to balance useful architectural information and irrelevant background noise in the skip connection concatenation. After two more 3×3 convolutions, batch normalizations, and ReLU activation processes, the upsampled output is fed into the next layer decoder.
[0044] The first three decoder layers replace skip connections with an attention gating mechanism, and then undergo two 3×3 convolutions, batch normalization, and ReLU activation.
[0045] The network model finally obtains the probability distribution of a single channel through 1×1 convolution, and then passes it through a sigmoid function to obtain the final segmentation result. 0 represents the background, and 1 represents the extracted building.
[0046] Figure 3 This is a diagram of a two-way skip attention mechanism:
[0047] The dual-path skip attention mechanism in the fourth encoding layer consists of three repeated, unpadded 3x3 convolutions C(·) and batch normalization (BN). Each C(·) is activated using the ReLU function σ(·). The second convolutional feature output is scaled down by a max pooling D(·) with a stride of 2. Furthermore, the second convolutional feature... and the third convolutional feature The data is passed to the dual-channel skip attention mechanism module. The decoder section performs upsampling using double linear interpolation and halves the number of channels using repeated convolutions C(·) before passing it to the dual-channel skip attention mechanism module. In the dual-channel skip attention mechanism module, the upsampled block is first... Perform a 1×1 convolution operation, and then convolution it with the result of the 1×1 convolution block. and Additive fusion is performed. Next, the fused features are activated using the ReLU activation function and mapped to one dimension via a 1×1 convolution. Then, an attention map is generated using the Sigmoid function, by interpolating the generated attention coefficients with... and Multiplying these features yields more relevant and useful features for prediction. Finally, the obtained features are concatenated with the upsampled result and then fed into the decoder. The formula is as follows:
[0048]
[0049]
[0050] Figure 4 This is a global-local focus module diagram:
[0051] The global-local focusing module is placed in the output features of the fifth layer encoder. Then, before inputting into the global focusing module and the local focusing module, feature smoothing is performed using a smoothing block. The specific formula can be expressed as:
[0052]
[0053]
[0054] In the global focusing module, a multi-scale structure is employed to enhance the ability of convolutional operations to capture spatial relationships, and a more refined attention map is generated through 1×1 convolutions. Specifically, three pairs of depth strip convolutions are used to approximate standard depthwise convolutions with large kernels, with kernel sizes of 7, 11, and 21 for each branch, respectively. The specific formula can be expressed as:
[0055]
[0056] In the local focusing module, an encoder using an intrinsic dictionary is employed. The total spatial number of the intrinsic codebook represents the spatial size of the feature map, and the scaling factor represents the learnable visual center. Specifically, features are encoded through a set of convolutional layers and processed by a 1×1 convolutional layer after residual connections, which also includes batch normalization layers and the ReLU activation function. Through the above operations, the encoded features are input into the codebook, and a set of scaling factors is used to map the corresponding positional information. After obtaining the codebook, fully connected layers are further used to predict features of salient key classes, and activation is performed using the sigmoid activation function. Subsequently, channel multiplication and channel addition are performed in conjunction with the output of the smoothing block. The specific formula can be expressed as:
[0057]
[0058] Figure 5 The image shows the results of a remote sensing image building extraction method based on a dual-path skip attention mechanism, used to demonstrate the building extraction effect based on the method of this invention.
[0059] Example 2
[0060] This embodiment relates to a computer-readable storage medium storing a program that, when executed by a processor, implements the remote sensing image building extraction method based on a dual-path skip attention mechanism as described in Embodiment 1.
[0061] Example 3
[0062] This embodiment relates to a computing device, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, it implements the remote sensing image building extraction method based on a dual-path skip attention mechanism of Embodiment 1.
[0063] The above-described embodiments are merely one implementation of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention.
Claims
1. A method for building extraction from remote sensing images based on a dual-path jump attention mechanism, characterized in that: A dual-path skip attention mechanism was used instead of skip connections. By incorporating a U-shaped encoder-decoder architecture with a global-local focusing module, more accurate extraction of buildings from remote sensing images was achieved. The network model used includes: encoder, decoder, dual-path skip attention mechanism, global-local focusing module, and attention gating mechanism. The specific steps include: Collect remote sensing datasets and corresponding mask label data, and train the constructed DSAUNet model using the datasets to obtain the trained DSAUNet model; Input remote sensing image data to be processed, and analyze and calculate it through the trained DSAUNet model to obtain building mask images; The constructed DSAUNet model includes: Based on the U-shaped encoder-decoder architecture, a dual-path skip attention mechanism is used to replace the skip connection of the fourth layer, the first three layers use an attention gating mechanism, and a global-local focusing module is integrated before the decoder; The dual-path skip attention mechanism includes: The output of the fourth coding layer is processed using 3×3 convolution, batch normalization, and ReLU activation function, and then passed through an attention gating mechanism. Finally, it is concatenated with the upsampled result of the decoding layer. The dual-path skip attention mechanism in the fourth encoding layer consists of three repeated, unpadded 3x3 convolutions C(·) and batch normalization (BN); each C(·) is activated using the ReLU function σ(·); the second convolutional feature is scaled down by a max pooling D(·) with a stride of 2; and the second convolutional feature... and the third convolutional feature The data is passed to the dual-channel skip attention mechanism module; the decoder uses double linear interpolation for upsampling and halves the number of channels using repeated convolution C(·) before being passed to the dual-channel skip attention mechanism module; in the dual-channel skip attention mechanism module, the upsampled block is first processed... Perform a 1×1 convolution operation, and then convolution it with the result of the 1×1 convolution block. and Additive fusion is performed; then, the fused features are activated using the ReLU activation function and mapped to one dimension through a 1×1 convolution; finally, an attention map is generated using the Sigmoid function, and the generated attention coefficients are respectively compared with... and Multiply the samples to obtain features useful for prediction; finally, concatenate the obtained features with the upsampling results and input them into the decoder. The global-local focusing module includes: The input features are first processed by smoothing blocks, then passed through the global focusing module and the local focusing module respectively, and finally output after channel concatenation; In the global focusing module, a multi-scale structure is adopted, using three pairs of depth strip convolutions, with the kernel size of each branch set to 7, 11 and 21 respectively, and 1×1 convolutions are used; In the local focusing module, an encoder using an intrinsic dictionary is employed to input the encoded features into the codebook and use a set of scaling factors to map the corresponding positional information; then a fully connected layer is used and activated by the sigmoid function; finally, channel multiplication and channel addition are performed by combining the output of the smoothing block.
2. The method according to claim 1, characterized in that, The collection of remote sensing datasets and corresponding mask label data includes: The collected remote sensing images and mask label data were cropped to a size of 512×512; data augmentation was performed to expand the dataset, including horizontal and vertical flipping, rotation, and blurring, and the training set, validation set, and test set were randomly divided in a 3:1:1 ratio.
3. The method according to claim 1, characterized in that, The trained DSAUNet model includes: Input the training data into the DSAUNet model, set the optimizer to AdamW, and the initial learning rate to 1e-4; calculate the loss and update the network parameters using BCELoss supervision; select the best training weights through the validation set and save the weight file.
4. The method according to claim 1, characterized in that, The analysis and computation performed using the trained DSAUNet model include: Load the saved weight file, extract buildings from the test set images, obtain the final building extraction mask, and perform qualitative and quantitative analysis.
5. A computer-readable storage medium, characterized in that, It stores a program that, when executed by a processor, implements the remote sensing image building extraction method based on the dual-path skip attention mechanism as described in any one of claims 1-4.
6. A computing device comprising a memory and a processor, wherein, The memory stores executable code, and when the processor executes the executable code, it implements the remote sensing image building extraction method based on the dual-path skip attention mechanism as described in any one of claims 1-4.