Road extraction method from remote sensing images based on asymmetric codec

Abstract

The present invention belongs to the field of artificial intelligence technology and discloses a method for extracting roads from remote sensing images based on an asymmetric codec. First, the encoder is improved and a hybrid dilated convolution module is designed to expand the encoder's receptive field and achieve deeper feature extraction. Secondly, a dual-branch decoder is proposed. One branch uses asymmetric convolution to extract linear features of the road, effectively solving the problem of road breakage caused by occlusion; the other branch uses standard convolution to learn segmentation, restore spatial details, and generate smooth road boundaries. The hybrid dilated convolution module designed in the present invention can expand the receptive field of the convolutional neural network and improve the quality of predicted roads. Secondly, the asymmetric convolution module is used to extract linear features of the road, aggregate remote context, and improve the topological connectivity of the road when the road is blocked by trees, buildings, shadows, etc.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a remote sensing image road extraction method based on an asymmetric codec. Background Art

[0002] Roads are important man-made features that form the backbone of modern transportation infrastructure. The timely updating of Geographic Information Systems (GIS) relies on the generation of high-precision road maps. Applications such as autonomous driving, urban planning, and emergency response also require accurate road information. Road maps are typically constructed using radar point clouds, GPS trajectories, or manual annotation. These methods are time-consuming and inefficient, making timely road map updates difficult in today's rapidly changing traffic environment. The continuous launch of my country's remote sensing satellite fleet in recent years has facilitated the acquisition of large-scale, high-resolution remote sensing imagery. Therefore, automatic road extraction from remote sensing imagery is a promising approach. However, due to the complexity of remote sensing imagery and the structural characteristics of roads themselves, extraction results remain unsatisfactory. Occlusion caused by trees, buildings, and shadows can lead to fragmented extraction results. Compared to other surface objects such as buildings, roads can be viewed as narrow, continuous areas with similar textures and spectra, which are easily interrupted, resulting in poor connectivity.

[0003] To model road topology and geometry, traditional road extraction algorithms typically combine edge detection, region clustering, and filtering algorithms to achieve integrated results, as a single algorithm struggles to fit complex road structures. These algorithms can be categorized as supervised or unsupervised, depending on whether labeled data is required. Typical supervised learning algorithms, such as support vector machines (SVMs), classify roads based on their spectral values ​​and local linear features. Early researchers attempted to use neural network models for road detection in high-resolution aerial imagery. However, these networks were overly simple, consisting of only a single hidden layer, making them difficult to fit complex road features. Among unsupervised learning algorithms, template matching-based, object-oriented methods are widely used for road extraction in remote sensing images. These algorithms rely on artificially designed prior knowledge of roads and are rigorously mathematically proven. However, their drawbacks include the need for parameter adjustment and the resulting error accumulation.

[0004] In recent years, due to the significant success of convolutional neural networks (CNNs) in computer vision tasks, CNN-based transfer learning has seen rapid progress in the field of road extraction from remote sensing images. Some research approaches have considered road extraction as a binary semantic segmentation task, and CNN models with a fully convolutional neural network (FCN) architecture have been widely used for this task. Unlike CNNs, FCNs remove the fully connected layers of CNNs to preserve spatial information. Instead, they use deconvolution or bilinear upsampling to restore the resolution of feature maps, and finally perform pixel-by-pixel classification using a 1×1 convolutional layer.

[0005] Previous studies mostly used symmetric encoder-decoder architectures, which are time-consuming and introduce more noise. Since roads account for a small proportion of remote sensing images, the impact of noise will be greater, so skip connections with low-level features are unnecessary. Summary of the Invention

[0006] Previous studies mostly used symmetric codec architectures, which are time-consuming and introduce more noise. Since roads account for a small proportion of remote sensing images, the impact of noise will be greater, so jump connections with low-level features are unnecessary. The present invention proposes a remote sensing image road extraction method based on an asymmetric codec, in which the constructed asymmetric architecture avoids this problem. Based on the symmetric codec architecture, the encoder is first improved and a hybrid void convolution module is designed to expand the encoder's receptive field and achieve deeper feature extraction. Secondly, a dual-branch decoder is proposed. One branch uses asymmetric convolution to extract the linear features of the road, effectively solving the problem of road breakage caused by occlusion; the other branch uses standard convolution to learn segmentation, restore spatial details, and generate smooth road boundaries. The hybrid void convolution module designed by the present invention can expand the receptive field of the convolutional neural network and improve the quality of road prediction. Secondly, the asymmetric convolution module is used to extract the linear features of the road, aggregate remote context, and improve the topological connectivity of the road when the road is blocked by trees, buildings, shadows, etc.

[0007] In order to achieve the above object, the present invention adopts the following technical solutions:

[0008] A method for extracting roads from remote sensing images based on an asymmetric codec, comprising:

[0009] Step 1: Build a deep learning model AEDNet for extracting roads from remote sensing images. The AEDNet is designed with an asymmetric architecture, including an encoder and a dual-branch decoder.

[0010] Step 2: Extract roads from remote sensing images based on the constructed AEDNet.

[0011] Furthermore, the encoder includes ResNet and a hybrid dilated convolution module.

[0012] Furthermore, the ResNet adopts ResNet101 pre-trained on ImageNet. In the hybrid dilated convolution module, the number of convolution layers N=3, the convolution kernel K=3, and the dilation rates are [1, 2, 3] respectively.

[0013] Furthermore, the decoder directly adds and fuses the features extracted by ResNet and the hybrid void convolution module, and then the decoder is divided into two branches: one branch uses standard convolution to learn segmentation; the other branch uses an asymmetric convolution module to extract linear features of the road.

[0014] Furthermore, the asymmetric convolution module includes a cascade of asymmetric convolutions with 1×3 and 3×1 kernels and a standard convolution with a 3×3 square kernel.

[0015] Furthermore, the loss calculation of the AEDNet adopts a combination of binary cross entropy loss and sieve coefficient.

[0016] Compared with the prior art, the present invention has the following beneficial effects:

[0017] Based on the symmetric encoding and decoding architecture, the present invention first improves the encoder and designs a hybrid void convolution module to expand the encoder's receptive field and achieve deeper feature extraction. Secondly, a dual-branch decoder is proposed. One branch uses asymmetric convolution to extract the linear features of the road, effectively solving the problem of road breakage caused by occlusion; the other branch uses standard convolution to learn segmentation, restore spatial details, and generate smooth road boundaries. The hybrid void convolution module designed by the present invention can expand the receptive field of the convolutional neural network and improve the quality of predicted roads. Secondly, the asymmetric convolution module is used to extract the linear features of the road and aggregate remote context, thereby improving the topological connectivity of the road when the road is blocked by trees, buildings, shadows, etc.

[0018] Experimental results on two benchmark datasets (DeepGlobe dataset and Massachusetts dataset) show that the designed road extraction method achieves significant improvements in IoU and F1 indicators, and also outperforms other algorithms in the connectivity indicator APLS. A large number of experimental results on the two datasets verify the effectiveness and robustness of the proposed method. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1This is a basic flow chart of a method for extracting roads from remote sensing images based on an asymmetric codec according to an embodiment of the present invention;

[0020] Figure 2 Schematic diagram of the overall framework of the deep learning model AEDNet constructed in an embodiment of the present invention;

[0021] Figure 3 Schematic diagram of the symmetric encoder-decoder architecture;

[0022] Figure 4 Schematic diagram of the asymmetric convolution module architecture according to an embodiment of the present invention;

[0023] Figure 5 This is an example diagram of the visual comparison results of an embodiment of the present invention. DETAILED DESCRIPTION

[0024] The present invention will be further explained below with reference to the accompanying drawings and specific embodiments:

[0025] like Figure 1 As shown, a remote sensing image road extraction method based on an asymmetric codec includes:

[0026] Step 1: Build a deep learning model AEDNet for extracting roads from remote sensing images. The AEDNet is designed with an asymmetric architecture, including an encoder and a dual-branch decoder.

[0027] Step 2: Extract roads from remote sensing images based on the constructed AEDNet.

[0028] Furthermore, the encoder includes ResNet and a hybrid dilated convolution module.

[0029] Furthermore, the ResNet adopts ResNet101 pre-trained on ImageNet. In the hybrid dilated convolution module, the number of convolution layers N=3, the convolution kernel K=3, and the dilation rates are [1, 2, 3] respectively.

[0030] Furthermore, the decoder directly adds and fuses the features extracted by ResNet and the hybrid void convolution module, and then the decoder is divided into two branches: one branch uses standard convolution to learn segmentation; the other branch uses an asymmetric convolution module to extract linear features of the road.

[0031] Furthermore, the asymmetric convolution module includes a cascade of asymmetric convolutions with 1×3 and 3×1 kernels and a standard convolution with a 3×3 square kernel.

[0032] Furthermore, the loss calculation of the AEDNet adopts a combination of binary cross entropy loss and sieve coefficient.

[0033] The details are as follows:

[0034] 1 AEDNet Model

[0035] 1.1 Overall Framework

[0036] This paper proposes a deep learning model AEDNet model for extracting roads in remote sensing images. Figure 2 The asymmetric architecture design of the AEDNet model is presented, which consists of an encoder and a dual-branch decoder.

[0037] Encoder: Due to its powerful feature learning ability, deep residual network (ResNet) is widely used in various computer vision tasks. Therefore, AEDNet uses ResNet101 pre-trained on ImageNet as the feature extraction backbone of the encoder, and the size of its output feature map is 1 / 8 of the input image. Considering that the roads in remote sensing images are narrow and long with large spans, expanding the receptive field of CNN can establish the long-range context relationship of the road, improve the topological connectivity of the road, and generate a more complete road map, such as Figure 2 As shown in the figure, the present invention designs a mixed dilated convolution (MDC) module to expand the receptive field of CNN. In order to avoid the "grid effect", that is, continuous dilated convolution may cause some feature points to be unable to participate in the calculation, assuming that the convolution kernel size is K×K, the dilation rates are [r1,...r i ,...r n ], according to the literature [WANG P, CHEN P, YUAN Y, et al. Understanding Convolution for Semantic Segmentation; proceedings of the 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), F 12-15 March 2018, 2018[C].], M2≤K should be satisfied, where:

[0038]

[0039] In the MDC module designed by the present invention, the number of convolution layers N=3, the convolution kernel K=3, and the void ratios are [1, 2, 3] respectively. According to formula (1), M2=2≤K=3, which meets the requirements of formula 1.

[0040] Decoder: Although traditional symmetric codec architectures (such as Figure 3 The skip-connection feature fusion strategy in

[15] can enable the decoder to reuse more low-level detail information. However, recent research literature shows that aggregating low-level features does not contribute much to binary classification tasks, but instead increases noise and computational cost. Figure 2 As shown in Figure 1, the AEDNet decoder directly adds and fuses the features extracted by the ResNet and MDC modules, rather than using skip connections. The decoder then splits into two branches: one uses standard convolution to learn segmentation; the other uses the asymmetric convolution (AC) module designed in this paper, inspired by the fact that asymmetric convolution better conforms to road shape, to extract linear features of the road. This efficiently aggregates long-range context and effectively addresses the problem of fragmented extraction results caused by occlusion.

[0041] Loss function: Road extraction from remote sensing images based on CNN is essentially a binary classification task, considering only two categories: road and non-road. Binary cross entropy (BCE) loss is commonly used for binary classification tasks. In addition, due to the class imbalance problem, that is, the number of road pixels is far less than that of non-road pixels, this study considers using the sieve coefficient (DICE) loss to alleviate this problem. Therefore, the loss of AEDNet is defined as:

[0042] L loss =L BCE +α(1-L dice ) (2)

[0043] Among them L BCE is the binary cross entropy loss, L dice is the Dice coefficient, which is defined as:

[0044]

[0045]

[0046] where y i ∈[0,1] is the value of the label corresponding to the i-th pixel, N is the total number of pixels, p i ∈(0,1) is the probability value predicted by the model.

[0047] 1.2 Asymmetric Convolution Module

[0048] Most convolutions in CNN architectures have square kernels because the target features to be extracted are mostly blocky. However, roads in remote sensing images are long, narrow, and continuously distributed, and are easily obscured by trees, buildings, shadows, etc. CNNs with square convolution kernels are generally unable to recognize these obscured areas because they limit their ability to capture long-range context. Asymmetric convolutions, such as 1×N or N×1, have long, narrow kernels that better conform to the shape of the road. They efficiently aggregate long-range dependencies in road areas along a single spatial dimension and prevent irrelevant areas from interfering with feature learning.

[0049] like Figure 4 As shown in Figure 2, the present invention proposes an asymmetric convolution (AC) module. Input feature map (where H, W, and C represent height, width, and number of channels, respectively) is first fed into a 1×1 convolution to reduce the number of channels to C / 4 of the original. Then, an innovative feature extraction structure is constructed: a cascade of asymmetric convolutions with 1×3 and 3×1 kernels and a standard convolution with a 3×3 square kernel. Because roads in remote sensing images are long and narrow, asymmetric convolutions with long and narrow kernels can efficiently extract linear features of roads in both horizontal and vertical directions, while the 3×3 standard convolution extracts the main features of the road. Finally, as Figure 4 As shown in the figure, the three feature maps extracted by the cascaded convolution blocks are channel-merged, and then the feature maps are restored to the same size as the input feature map X through up-sampling and 1×1 convolution.

[0050] 2 Experiments

[0051] 2.1 Dataset

[0052] For the convenience of description, the method of the present invention is referred to as the AEDNet algorithm. In order to evaluate the performance of the AEDNet algorithm, the present invention conducted experiments on two popular public benchmarks: DeepGlobe and Massachusetts datasets.

[0053] Massachusetts Dataset: Contains 1,171 remote sensing images with a resolution of 1,300 × 1,300 and corresponding pixel-level annotations, covering over 2,600 square kilometers of Massachusetts. The images primarily depict urban areas. 1,108 images are used for training, 14 for validation, and 49 for testing. Images are cropped to 650 × 650 to augment the dataset.

[0054] DeepGlobe: This dataset provides 1024×1024 remote sensing images with a resolution of 50 cm / pixel and corresponding pixel-level annotations. This dataset was collected in Thailand, India, and Indonesia. Unlike the Massachusetts dataset, it includes not only urban areas but also rural areas, wilderness, coastal areas, and tropical rainforests. The dataset contains 6,226 images, divided into 4,696 for training and 1,530 for testing. Images are cropped to 512×512 to expand the dataset.

[0055] 2.2 Evaluation Metrics

[0056] The present invention uses semantic segmentation pixel-level evaluation indicators: Intersection over Union (IoU), F1 score (F-measure, F1) and connectivity indicator: Average Path Length Similarity (APLS) to evaluate the correctness and topological connectivity of the road prediction algorithm. The pixel-level evaluation indicators are defined as:

[0057]

[0058]

[0059] Where TP (true-positive) represents the number of pixels correctly predicted as roads; FP (false-positive) represents the number of background pixels classified as roads; TN (true-negative) represents the number of pixels correctly predicted as background; FN (false-negative) represents the number of road pixels classified as background. The connectivity index APLS is defined as:

[0060]

[0061] For the prediction results and corresponding label V, node a,b∈V, L(a,b) and Represents V and a→b, The path distance. N is the number of different paths.

[0062] 2.3 Experimental details

[0063] In this study, the optimizer uses the Stochastic Gradient Descent (SGD) algorithm with a batch size parameter of 32, and its momentum and weight decay coefficient parameters are set to 0.9 and 5×10 -4 , the initial learning rate (Learning rate, lr) is 0.02, using the "poly" strategy The learning rate was gradually reduced with power = 9. AEDNet was implemented using the PyTorch machine learning framework and trained on four NVIDIA RTX A40 GPUs with 48GB of video memory. During training, images in both datasets were cropped to 512×512. Other data augmentation strategies included random horizontal flipping, random rotation, and random Gaussian blurring to improve the model's generalization capabilities.

[0064] 2.4 Experimental comparison

[0065] To validate the effectiveness of our proposed asymmetric encoder-decoder network (AEDNet), we compared two classic algorithms based on symmetric encoder-decoder architectures, ResNet and D-LinkNet, on two popular benchmarks: DeepGlobe and Massachusetts road extraction datasets. To ensure fairness, all compared algorithms used the same data preprocessing and training strategies.

[0066] Quantitative Comparison on the Massachusetts Dataset: Table 1 lists quantitative comparative experiments on the Massachusetts dataset, using FCN as the baseline for comparison, constructed using ResNet101. The table shows that the AEDNet algorithm outperforms other algorithms in both pixel-level and connectivity metrics. For example, AEDNet achieves an F1 score of 78.64% and an IoU score of 65.37%, respectively, compared to the next-best D-LinkNet algorithm. Regarding road connectivity, the APLS metric, used to assess the topological connectivity of roads, shows that AEDNet achieves a 2.18% improvement over D-LinkNet. Considering the relatively small proportion of roads in remote sensing images, these improvements based on these metrics are significant. Because AEDNet's asymmetric encoding / decoding architecture avoids noise, it can more accurately extract roads from remote sensing images. Furthermore, the proposed asymmetric convolutional block (AC) further improves road topological connectivity.

[0067] Quantitative comparison of DeepGlobe dataset: Table 2 lists the quantitative comparative experiments on the DeepGlobe dataset. The results show that the AEDNet proposed in this invention has good robustness on different datasets. The F1 score of AEDNet is 79.32%, which is better than other methods and exceeds the second-place algorithm by 2.86%. AEDNet also achieved the best APLS score. The roads in the DeepGlobe dataset are mainly in rural areas, with a relatively sparse distribution. The spectral characteristics are similar to those of the adjacent areas, and there are many obstructions such as trees and buildings. The extraction accuracy of the algorithm based on the symmetric encoder-decoder architecture is not high. The qualitative experimental results in Table 2 show that AEDNet is also suitable for roads in rural areas. The roads in the Massachusetts dataset are mainly in urban areas and are relatively densely distributed. Combined with the results on the Massachusetts dataset, it is shown that the AEDNet proposed in this invention is robust to roads in different regions.

[0068] Table 1 Comparison results on the Massachusetts dataset

[0069]

[0070] Table 2 Comparison results on DeepGlobe dataset

[0071]

[0072] Qualitative comparison: The qualitative comparison results of AEDNet and other methods are shown in Figure 5 As shown in the figure, it shows 4 examples on the DeepGlobe and Massachusetts datasets. The 1st-2nd row and the 3rd-4th row are some experimental results on the DeepGlobe and Massachusetts datasets, respectively. Black represents TN, white represents TP, green represents FN, and red represents FP. Figure 5 As can be seen from the figure, the roads extracted by AEDNet are closest to the corresponding labels. Compared with other algorithms, there are very few false positives, which is a significant improvement. Figure 5 The first and second rows in the figure show that the roads extracted by other methods have many breaks, that is, more green areas, while the roads extracted by AEDNet have better topological connectivity and fewer green areas. It is worth noting that compared with AEDNet, ResUNet has over-prediction in some areas, such as Figure 5The prediction results of the ResUNet algorithm in the second row show a large number of red areas, that is, some non-roads are predicted as roads. The reason is that the receptive field of the ResUNet model is small. In contrast, AEDNet embeds the hybrid dilated convolution MDC module designed by the present invention, which uses multiple dilated convolutions with different dilation rates to expand the receptive field and can efficiently utilize multi-scale context information, thus achieving better prediction results. Even in some complex scenarios, such as Figure 5 In the third and fourth rows, AEDNet is superior to other algorithms, with fewer green areas and good road connectivity. Figure 5 The visualization results and a large number of experiments on two road extraction benchmark datasets show that the AEDNet proposed in this paper demonstrates significant superiority and robustness in the task of road extraction from remote sensing images.

[0073] In summary, this paper investigates a CNN-based road extraction method for remote sensing images and proposes AEDNet, based on an asymmetric codec architecture. Previous studies have mostly used symmetric codec architectures, which are time-consuming and introduce more noise. Since roads account for a small proportion of remote sensing images, the impact of noise is greater, making skip connections to low-level features unnecessary. The asymmetric architecture proposed in this paper avoids this problem. Based on this, the present invention considers the wide coverage and narrow distribution of roads in remote sensing images. First, a hybrid dilated convolutional (MDC) module is designed to expand the receptive field of the CNN and improve the quality of road prediction. Second, an asymmetric convolutional (AC) module is used to extract linear features of roads and aggregate long-range context, which improves topological connectivity when roads are obscured by trees, buildings, shadows, and other factors. Experimental results on two benchmark datasets (DeepGlobe and Massachusetts) demonstrate that the designed road extraction method achieves significant improvements in the IoU and F1 metrics, and also outperforms other algorithms in the connectivity metric APLS. Extensive experimental results on both datasets validate the effectiveness and robustness of the AEDNet algorithm.

[0074] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for extracting roads from remote sensing images based on an asymmetric codec, characterized in that: include: Step 1: Build a deep learning model AEDNet to extract roads in remote sensing images; The AEDNet is designed with an asymmetric architecture, including an encoder and a dual-branch decoder; the encoder includes ResNet and a hybrid dilated convolution module; the ResNet uses ResNet101 pre-trained on ImageNet, and the number of convolution layers in the hybrid dilated convolution module is N =3, convolution kernel K =3, with dilation rates of [1, 2, 3] respectively. The decoder directly adds and fuses the features extracted by the ResNet and hybrid dilated convolution modules. The decoder then splits into two branches: one branch uses standard convolution to learn segmentation; the other branch uses an asymmetric convolution module to extract linear features of the road. The asymmetric convolution module includes a cascade of asymmetric convolutions with 1×3 and 3×1 kernels, respectively, and a standard convolution with a 3×3 square kernel. Step 2: Extract roads from remote sensing images based on the constructed AEDNet.

2. The method for extracting roads from remote sensing images based on an asymmetric codec according to claim 1, wherein: The loss calculation of the AEDNet adopts a combination of binary cross entropy loss and sieve coefficient.

Images