Improved remote sensing image road extraction method based on double encoders

Abstract

The invention discloses an improved remote sensing image road extraction method based on double encoders. The method comprises the following steps: preparing a data set; building a remote sensing image road extraction network; training a remote sensing image road extraction network; and testing the remote sensing image road extraction network. According to the method, feature extraction is carried out on an encoder by using double encoders combining Swin Transformer and pre-trained Resnet-34 so as to improve the connectivity of road extraction, and multi-scale feature representation is realized by combining a cascaded parallel cavity convolution block, so that fragmentization of extracted roads is reduced. According to the invention, a spatial pixel module is designed at the encoder, and spatial information of pixel-level features is captured, so that inaccurate semantic information caused by shielding is relieved. And the global modeling capability of the Swin Transform due to the limitation of a window mechanism is enhanced. According to the invention, the complete structure of the road network is maintained, and the connectivity and integrity of the road are enhanced.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image segmentation and deep learning, and in particular to a remote sensing image road extraction method based on a dual encoder improvement. Background Art

[0002] As an important part of geographic information data, road data plays a key role in urban planning, autonomous driving, vehicle navigation, and geographic information system construction. However, manual annotation of road data is a time-consuming and arduous task, especially when there is a large area of ​​road network coverage in the image. Therefore, these fields urgently need efficient automatic road extraction methods to provide accurate and reliable road information. The research on automatic road extraction methods can significantly save time and labor costs, and is of great significance to improving traffic safety, supporting autonomous driving, and promoting the development of smart cities. In recent years, various automatic road extraction methods have been proposed, which are mainly divided into traditional methods and deep learning-based methods.

[0003] Traditional road extraction methods, such as threshold segmentation, edge detection, texture analysis, and mathematical morphology algorithms, use shallow features (such as grayscale, edge, spectrum, texture, and geometric features) to extract roads from remote sensing images. They are usually limited by fixed parameters. When processing large-area, high-resolution satellite images, these methods cannot fully capture road features at different scales, and are often inefficient and ineffective.

[0004] With the continuous development of deep learning technology in image segmentation, the use of deep learning to process road extraction methods has become mainstream. Remote sensing image road extraction methods based on convolutional neural networks (CNN) have attracted the interest of researchers in the field of remote sensing image processing. A series of advanced deep convolutional neural networks have been applied to various remote sensing image segmentation tasks. Compared with traditional road extraction methods, remote sensing image road extraction methods based on deep convolutional neural networks have the advantages of high automation, automatic learning and accurate extraction of road features from large-scale remote sensing images, strong adaptability and generalization, and have received extensive attention and research. These methods overcome the shortcomings of traditional methods and perform better than traditional methods. However, roads may be partially blocked by buildings, trees or other objects in remote sensing images, and the presence of shadows will cause changes in the brightness and color of the road area. These occlusions and shadows will cause discontinuities in the road area. To solve this problem, algorithms are required to be able to comprehensively utilize global information to identify and fill in missing parts. However, the above-mentioned CNN-based methods mainly focus on the extraction of local features, and their limited receptive fields limit the ability to recognize road integrity, resulting in inaccurate segmentation results.

[0005] In recent years, Transformer-based methods have been used in computer vision and have achieved good results. Unlike previous CNN-based methods, Transformer has a strong ability to capture long-distance dependencies, has stronger contextual semantic relevance and a wider receptive field, and shows excellent transferability in downstream tasks. However, the road extraction task faces challenges brought by complex scenes and background interference such as buildings, trees, and vehicles. These interfering objects are similar to roads in texture and color, making it difficult to distinguish. In order to improve the accuracy of distinction, the key is to more effectively utilize the correlation information and channel information between pixels in the feature map to accurately segment the road. Although Transformer-based methods have significant advantages in capturing long-distance dependencies, they have limitations in capturing local details and integrating spatial pixel information and channel information, resulting in poor results in distinguishing roads from interfering objects and poor connectivity of the extracted roads. Summary of the invention

[0006] In summary, in order to solve the problem of automatic road extraction in high-resolution remote sensing images, the present invention proposes a remote sensing image road extraction method based on an improved dual encoder, aiming to improve the connectivity and integrity of roads and maintain the integrity of the road network to the greatest extent.

[0007] In order to achieve the above-mentioned object of the invention, the present invention adopts the following technical scheme:

[0008] A remote sensing image road extraction method based on dual encoder improvement includes the following steps:

[0009] A. Preparation of Dataset

[0010] Collect high-resolution remote sensing image data taken by remote sensing satellites and prepare a remote sensing image road extraction dataset according to the following steps:

[0011] A1. Label and crop the image

[0012] A2. Image Enhancement

[0013] B. Build a remote sensing image road extraction network

[0014] The remote sensing image road extraction network is a remote sensing image road extraction network improved by dual encoders. The overall architecture adopts an encoder-decoder architecture, and the encoder includes a main encoder and an auxiliary encoder. The main encoder is a ResNet block composed of ResNet-34; the auxiliary encoder is a SwinConv block composed of a Swin Transformer module and a spatial pixel module. Each time, the batch size of enhanced remote sensing images is input into the Conv block containing convolution and pooling operations and the first SwinConv block respectively, and the feature map output by the Conv block is input into the first ResNet block. The feature maps output by the first SwinConv block are respectively input into the first feature fusion module and the second SwinConv block, the feature maps output by the first ResNet block are input into the first feature fusion module, the feature maps output by the first feature fusion module are respectively input into the second ResNet block and the first jump connection unit, the feature maps output by the second SwinConv block are respectively input into the second feature fusion module and the third SwinConv block, the feature maps output by the second ResNet block are input into the second feature fusion module, the feature maps output by the second feature fusion module are respectively input into the third ResNet block and the second jump connection unit, the feature maps output by the third SwinConv block are respectively input into the third feature fusion module and the fourth SwinConv block, the feature maps output by the third ResNet block are input into the third feature fusion module, and the third feature fusion module The feature maps output by the block are respectively input into the fourth ResNet block and the third skip connection unit, the feature maps output by the fourth SwinConv block and the feature maps output by the fourth ResNet block are both input into the fourth feature fusion module, and the feature maps output by the fourth feature fusion module are input into the dilated convolution module; the feature maps output by the dilated convolution module are input into the first decoding block, the feature maps output by the first decoding block are fused with the feature maps from the third skip connection unit and then input into the second decoding block, the feature maps output by the second decoding block are fused with the feature maps from the second skip connection unit and then input into the third decoding block, the feature maps output by the third decoding block are fused with the feature maps from the first skip connection unit and then input into the fourth decoding block, and the feature maps output by the fourth decoding block are outputted through the Conv block containing transposed convolution and convolution operations to output a prediction map of the same size as the input remote sensing image.

[0015] C. Training remote sensing image road extraction network

[0016] The enhanced remote sensing image dataset prepared in step A is input into the remote sensing image road extraction network for training. During training, a composite loss function consisting of binary cross entropy loss and dice loss is used as the final loss function. The initial learning rate is set to 2e-4, and the adaptive moment estimation optimization algorithm is used to adaptively adjust the learning rate. When the learning rate is less than 5e-7, the training is stopped.

[0017] D. Testing the Remote Sensing Image Road Extraction Network

[0018] Select the road extraction network parameters with the smallest loss value on the training set from step C, and test the road extraction performance of the remote sensing image road extraction network on the test set.

[0019] Furthermore, the method for marking and cropping the image in step A1 includes the following steps:

[0020] Download the original remote sensing image dataset from the official website of remote sensing image dataset. The original remote sensing image dataset is a RGB three-band color remote sensing image dataset and annotate it using dataset annotation software. Then, the original remote sensing images and manually annotated label images are uniformly cropped to 1024×1024 pixel images in the same way to obtain a remote sensing image dataset with no less than 6,000 labeled images of 1024×1024 pixels, referred to as labeled remote sensing image dataset.

[0021] Furthermore, the image enhancement method in step A2 comprises the following steps:

[0022] An image enhancement operation is performed on the labeled remote sensing image dataset, and the image enhancement operation includes horizontal flipping, vertical flipping, diagonal flipping, color jittering and translation. The dataset after the image enhancement operation is called an enhanced remote sensing image dataset.

[0023] Furthermore, the method for building a remote sensing image road extraction network in step B comprises the following steps:

[0024] B1. Build a dual encoder

[0025] In the encoder module, Swin Transformer is fused with ResNet-34 pre-trained on the ImageNet dataset as a dual encoder to extract feature information. ResNet-34 is the main encoder, and the SwinConv block, which is a combination of SwinTransformer and spatial pixel module, is the auxiliary encoder. A feature fusion module is designed in the part where the main encoder and the auxiliary encoder are fused; in the auxiliary encoder SwinConv block, a spatial pixel module across the W-Trans and SW-Trans blocks in the Swin Transformer is designed.

[0026] B2. Introducing a dilated convolution module into the central part of the remote sensing image road extraction network

[0027] The dilated convolution module is composed of dilated convolutions with different dilated rates stacked in a cascade mode. The dilated rates of the stacked dilated convolutions are 1, 2, 4, and 8, respectively, and the corresponding receptive fields are 3, 7, 15, and 31, respectively.

[0028] B3. Design a spatial pixel module in the Swin Transformer part of the encoder of the remote sensing image road extraction network

[0029] When the image enters the SwinConv block, for the features in the nth SwinConv block, first convert the input feature s of SwinTransformer l-1 Reshape into z, where c1=2 n-1 C1,h=(H / (2 n+1 )) and w=(W / (2 n+1 )).

[0030] The reshaped feature z is sent to a dilated convolution with a size of 3×3 and a dilation rate of 2, and the number of channels becomes c1 / 2. Then, the statistical features in the three dimensions of height h, width w and channel c are obtained through the global average pooling operation. Specifically, the element calculations in the three dimensions are expressed as follows:

[0031]

[0032] In the formula, GAP h , GAP w and GAP c They are global average pooling in three dimensions: height h, width w, and channel c. is the dilated convolution layer, batch normalization, and GELU activation. The tensors in three dimensions are v h 、v w and v c ,,v h 、v w and v c are the weights of feature z converging on height h, width w, and channel c, respectively. and The three weight information are combined to obtain the spatial attention information I, and the calculation formula is as follows:

[0033]

[0034] in, represents matrix multiplication,

[0035] Finally, the spatial attention information I is combined with the Swin Transformer block s l+1 The output of the SwinConv block is added to obtain the output feature L, which is matched by a 1×1 convolution layer. l+1 The output feature L is expressed as follows:

[0036]

[0037] in, represents matrix addition, Represents a 1×1 convolutional layer, batch normalization, and GELU activation.

[0038] B4. Design a feature fusion module in the encoder part of the remote sensing image road extraction network

[0039] Assume that the outputs of the main encoder and the auxiliary encoder at the nth stage are A n With S n . n Input to the 1×1 convolution layer to change the size and obtain features

[0040] First, apply the maximum pooling to calculate the statistical features and the fully connected layer to obtain At the same time, the global weight is calculated using average pooling and put into a fully connected layer. The result is P A This process is expressed as follows:

[0041]

[0042] Where σ represents the ReLu function.

[0043] Then, use P A Optimize P for each channel M , which is expressed as follows:

[0044] P = δ(FC(P M ⊙P A )) (5)

[0045] In the formula, δ represents the Sigmoid function and ⊙ represents element-wise multiplication. The channel-dependent weight P is used with the output A of the main encoder. n Multiply them together to get the extracted features.

[0046] Finally, the output feature F of the feature fusion module n It is expressed as follows:

[0047]

[0048] Compared with the prior art, the present invention has the following beneficial effects:

[0049] 1. In the encoder part, the present invention uses a dual encoder combining Swin Transformer and pre-trained Resnet-34 for feature extraction to improve the connectivity of road extraction, and combines a cascaded parallel hole convolution block to realize multi-scale feature representation and reduce the fragmentation of extracted roads.

[0050] 2. The present invention designs a spatial pixel module in the Swin Transformer part of the encoder to capture the spatial information of pixel-level features, thereby alleviating the inaccuracy of semantic information caused by occlusion, and enhancing the global modeling capability of Swin Transformer limited by the window mechanism.

[0051] 3. The present invention designs a feature fusion module in the fusion part of the two encoders, extracts channel-related information from Swin Transformer as global clues to guide Resnet, and can effectively distinguish roads blocked by obstacles.

[0052] 4. In summary, the present invention not only maintains the complete structure of the road network, but also enhances the connectivity and integrity of the roads, and has broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 It is a structural diagram of the present invention.

[0054] Figure 2 It is the structural diagram of the hole convolution module.

[0055] Figure 3 This is a diagram of the SwinConv block and its internal module structure.

[0056] Figure 4 It is the structural diagram of the feature fusion module.

[0057] Figure 5 This is a comparison chart of the road extraction effects of the present invention and other road segmentation networks. DETAILED DESCRIPTION

[0058] The present invention will be further described below in conjunction with the accompanying drawings. Figure 1 As shown in FIG. 1 , a remote sensing image road extraction method based on a dual encoder improvement is shown in FIG. 1 . The specific steps are as follows:

[0059] Step 1: Collect remote sensing image data taken by remote sensing satellites. Download the original remote sensing image dataset from the official website of remote sensing image dataset. The original remote sensing image dataset is an RGB three-band color remote sensing image dataset.

[0060] Step 2: Annotation of remote sensing image dataset. Use remote sensing image annotation tools to annotate the road areas in the RGB three-channel color remote sensing images collected from remote sensing satellites, and obtain a label image with the same size as the original image. The label image is a black and white binary image, in which the road markings are black and the background is white.

[0061] Step 3: Image cropping. The original RGB color remote sensing image and its corresponding label image are cropped into a small image of 1024×1024 pixels in the same way, and a dataset of 6226 labeled remote sensing images of 1024×1024 pixels is obtained.

[0062] Step 4: Image enhancement. The 6226 cropped 1024×1024 pixel remote sensing images were enhanced by using operations such as horizontal flipping, vertical flipping, diagonal flipping, color jittering, and translation.

[0063] Step 5: Dataset division: The 6226 labeled remote sensing images after image enhancement are randomly divided into training and test sets in a ratio of 4:1, resulting in 4981 training images and 1245 test images.

[0064] Step 6: Build a remote sensing image road extraction network. The remote sensing image road extraction network is an improved remote sensing image road extraction network based on dual encoders. The overall architecture adopts an encoder-decoder architecture. The encoder includes a main encoder and an auxiliary encoder. The main encoder is a ResNet block composed of ResNet-34, and the auxiliary encoder is a SwinConv block composed of SwinTransformer and a designed spatial pixel module. The SwinConv block structure is as follows: Figure 3 In addition, it also includes a feature fusion module and a hole convolution block, as shown in Figure 4 and Figure 2As shown. The enhanced remote sensing image is input into the Conv block containing convolution and pooling operations and the first SwinConv block respectively, and the feature map output by the Conv block is input into the first ResNet block. The feature maps output by the first SwinConv block are input into the first feature fusion module and the second SwinConv block respectively, and the feature maps output by the first ResNet block are input into the first feature fusion module. The feature maps output by the first feature fusion module are input into the second ResNet block and the first jump connection part respectively, and the feature maps output by the second SwinConv block are input into the second feature fusion module and the third SwinConv block respectively, and the feature maps output by the second ResNet block are input into the second feature fusion module. The feature maps output by the second feature fusion module are input into the third ResNet block and the second jump connection part respectively, and the feature maps output by the third SwinConv block are input into the third feature fusion module and the fourth SwinConv block respectively, and the feature maps output by the third ResNet block are input into the third feature fusion module. The feature maps output by the block are respectively input into the fourth ResNet block and the third skip connection part, the feature maps output by the fourth SwinConv block and the feature maps output by the fourth ResNet block are both input into the fourth feature fusion module, and the feature maps output by the fourth feature fusion module are input into the dilated convolution module; the feature maps output by the dilated convolution module are input into the first decoding block, the feature maps output by the first decoding block are fused with the feature maps from the third skip connection part and then input into the second decoding block, the feature maps output by the second decoding block are fused with the feature maps from the second skip connection part and then input into the third decoding block, the feature maps output by the third decoding block are fused with the feature maps from the first skip connection part and then input into the fourth decoding block, and the feature maps output by the fourth decoding block are outputted through the Conv block containing transposed convolution and convolution operations to output a prediction map of the same size as the input remote sensing image.

[0065] Step 7: Train the remote sensing image road extraction network. The 4981 training images obtained after the division in Step 5 are input into the remote sensing image road extraction network, and training is performed simultaneously on two NVIDIA GeForce RTX 3090 graphics cards. The number of images processed for each training is set to 8. During training, a composite loss function consisting of binary cross entropy loss and dice loss is used as the final loss function. The initial learning rate is set to 2e-4, and the adaptive moment estimation optimization algorithm is used to adaptively adjust the learning rate. When the learning rate is less than 5e-7, the training is stopped. The road extraction network parameters with the smallest loss value on the training set are selected, and the road extraction performance of the remote sensing image road extraction network is tested on the test set.

[0066] Step 8: Perform performance test on the remote sensing image road extraction network. Input the 1245 test images obtained after the division in Step 5 into the trained remote sensing image road extraction network, and output the corresponding 1245 prediction images.

[0067] Step 9: Evaluation of prediction results of road extraction network based on remote sensing images. The prediction results are evaluated using evaluation indicators such as F1 score and intersection-over-union ratio, and a comprehensive evaluation is performed in combination with the visual comparison chart of the prediction results. Figure 5 As shown in the figure, some mainstream road extraction networks have shown good road extraction effects. However, these mainstream methods do not show good detection performance for roads in some complex scenes. The main reason is that they do not effectively extract and fuse the global and local information of the image and lack attention to channel information. Figure 5 The results of road extraction from these images using eight different methods are shown. Figure 5 The first and second rows of the image may be similar in color and texture to the surrounding farmland or land. This similarity makes it difficult for general road extraction algorithms to clearly distinguish road boundaries, while the remote sensing image road extraction network of the present invention can accurately distinguish between road and non-road areas compared to other methods. Figure 5 In the third row of , the road area is blocked by trees. In this case, the other seven methods cannot recognize or can only recognize part of the blocked road, while the remote sensing image road extraction network of the present invention can extract complete road information. Figure 5 The other seven methods miss more road information and the extracted roads are fragmented, while the remote sensing image road extraction network of the present invention can maintain a relatively complete road structure. For urban roads with rich spectral and texture features ( Figure 5 ), the other seven methods cannot completely extract the road structure, while the remote sensing image road extraction network of the present invention can accurately extract the road area.

[0068] The present invention is not limited to this embodiment, and any equivalent concepts or changes within the technical scope disclosed by the present invention are included in the protection scope of the present invention.

Claims

1. A remote sensing image road extraction method based on dual encoder improvement, characterized in that: The following steps are involved: A. Preparation of Dataset Collect high-resolution remote sensing image data taken by remote sensing satellites and prepare a remote sensing image road extraction dataset according to the following steps: A1. Label and crop the image; A2, Image enhancement; B. Build a remote sensing image road extraction network The remote sensing image road extraction network is a remote sensing image road extraction network improved by a dual encoder. The overall architecture adopts an encoder-decoder architecture. The encoder includes a main encoder and an auxiliary encoder. The main encoder is a ResNet block composed of ResNet-34. The auxiliary encoder is a SwinConv block composed of Swin The network consists of a Transformer module and a spatial pixel module; each time, the batch size of enhanced remote sensing images are input into the Conv block containing convolution and pooling operations and the first SwinConv block respectively, and the feature map output by the Conv block is input into the first ResNet block; the feature map output by the first SwinConv block is input into the first feature fusion module and the second SwinConv block respectively, the feature map output by the first ResNet block is input into the first feature fusion module, the feature map output by the first feature fusion module is input into the second ResNet block and the first jump connection unit respectively, the feature map output by the second SwinConv block is input into the second feature fusion module and the third SwinConv block respectively, the feature map output by the second ResNet block is input into the second feature fusion module, the feature map output by the second feature fusion module is input into the third ResNet block and the second jump connection unit respectively, the feature map output by the third SwinConv block is input into the third feature fusion module and the fourth SwinConv blocks are input into the third feature fusion module, the feature maps output by the third feature fusion module are respectively input into the fourth ResNet block and the third jump connection unit, the feature maps output by the fourth SwinConv block and the feature maps output by the fourth ResNet block are both input into the fourth feature fusion module, and the feature maps output by the fourth feature fusion module are input into the hole convolution module; the feature maps output by the hole convolution module are input into the first decoding block, the feature maps output by the first decoding block are fused with the feature maps from the third jump connection unit and then input into the second decoding block, the feature maps output by the second decoding block are fused with the feature maps from the second jump connection unit and then input into the third decoding block, the feature maps output by the third decoding block are fused with the feature maps from the first jump connection unit and then input into the fourth decoding block, and the feature maps output by the fourth decoding block are outputted through the Conv block containing transposed convolution and convolution operations to output a prediction map of the same size as the input remote sensing image; C. Training remote sensing image road extraction network The enhanced remote sensing image data set prepared in step A is input into the remote sensing image road extraction network for training; during training, a composite loss function consisting of binary cross entropy loss and dice loss is used as the final loss function; the initial learning rate is set to 2e-4, and the adaptive moment estimation optimization algorithm is used to adaptively adjust the learning rate. When the learning rate is less than 5e-7, the training is stopped; D. Testing the Remote Sensing Image Road Extraction Network Select the road extraction network parameters with the smallest loss value on the training set from step C, and test the road extraction performance of the remote sensing image road extraction network on the test set.

2. The method for extracting roads from remote sensing images based on a dual encoder improvement according to claim 1, characterized in that: The method for marking and cropping an image in step A1 comprises the following steps: The original remote sensing image dataset is downloaded from the official website of the remote sensing image dataset. The original remote sensing image dataset is an RGB three-band color remote sensing image dataset and is annotated using the dataset annotation software. Then, the original remote sensing images and the manually annotated label images are uniformly cropped into 1024×1024 pixel images in the same way to obtain a remote sensing image dataset of no less than 6000 labeled 1024×1024 pixel images, referred to as the labeled remote sensing image dataset.

3. The method for extracting roads from remote sensing images based on a dual encoder improvement according to claim 1, characterized in that: The image enhancement method described in step A2 comprises the following steps: An image enhancement operation is performed on a labeled remote sensing image dataset, wherein the image enhancement operation includes horizontal flipping, vertical flipping, diagonal flipping, color jittering, and translation; the dataset after the image enhancement operation is called an enhanced remote sensing image dataset.

4. The method for extracting roads from remote sensing images based on a dual encoder improvement according to claim 1, characterized in that: The method for building a remote sensing image road extraction network in step B comprises the following steps: B1. Build a dual encoder In the encoder module, Swin Transformer is fused with ResNet-34 pre-trained on the ImageNet dataset as a dual encoder to extract feature information; ResNet-34 is the main encoder, and the SwinConv block that combines Swin Transformer and spatial pixel module is the auxiliary encoder; a feature fusion module is designed in the part where the main encoder and the auxiliary encoder are fused; a spatial pixel module that spans the W-Trans and SW-Trans blocks in the Swin Transformer is designed in the auxiliary encoder SwinConv block; B2. Introducing a dilated convolution module into the central part of the remote sensing image road extraction network The dilated convolution module is formed by stacking dilated convolutions with different dilated ratios in a cascade mode. The dilated ratios of the stacked dilated convolutions are 1, 2, 4, and 8, respectively, and the corresponding receptive fields are 3, 7, 15, and 31, respectively. B3. Design a spatial pixel module in the Swin Transformer part of the encoder of the remote sensing image road extraction network When the image enters the SwinConv block, for the features in the nth SwinConv block, first convert the input feature s of SwinTransformer l-1 Reshape into z, where c1=2 n-1 C1,h=(H / (2 n+1 )) and w=(W / (2 n+1 )); The reshaped feature z is sent to a dilated convolution with a size of 3×3 and a dilation rate of 2, and the number of channels becomes c1 / 2. Then, the statistical features in the three dimensions of height h, width w and channel c are obtained through the global average pooling operation; specifically, the element calculations in the three dimensions are expressed as follows: In the formula, GAP h 、GAP w and GAP c They are global average pooling in three dimensions: height h, width w, and channel c; features is the dilated convolution layer, batch normalization, and GELU activation; the tensors in three dimensions are v h 、v w and v c ,,v h 、v w and v c are the weights of feature z converging on height h, width w, and channel c, respectively. and The three weight information are combined to obtain the spatial attention information I, and the calculation formula is as follows: in, represents matrix multiplication, Finally, the spatial attention information I is combined with the Swin Transformer block s l+1 The output of the SwinConv block is added to obtain the output feature L, which is matched by a 1×1 convolution layer. l+1 The dimension of the output feature L is as follows: in, represents matrix addition, represents a 1×1 convolutional layer, batch normalization, and GELU activation; B4. Design a feature fusion module in the encoder part of the remote sensing image road extraction network Assume that the outputs of the main encoder and the auxiliary encoder at the nth stage are A n With S n ; S n Input to the 1×1 convolution layer to change the size and obtain features First, apply the maximum pooling to calculate the statistical features and the fully connected layer to obtain At the same time, the global weight is calculated using average pooling and put into a fully connected layer. The result is P A This process is expressed as follows: Where, σ represents the ReLu function; Then, use P A Optimize P for each channel M , which is expressed as follows: P=δ(FC(P M ⊙P A )) (5) In the formula, δ represents the Sigmoid function, ⊙ represents element-wise multiplication; the channel-dependent weight P is used with the output A of the main encoder n Multiply to get the extracted features; Finally, the output feature F of the feature fusion module n It is expressed as follows:

Images