Remote sensing image road extraction method of cascade attention guidance context aggregation network
Through the cascading attention-guided context aggregation network (AGCANet), road information is extracted in remote sensing images, solving the problems of high computing costs and high model complexity, achieving efficient and accurate road extraction, and maintaining road connectivity and topological integrity.
Patent Information
- Application Number
- CN202510385245.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-28
- Publication Date
- 2025-08-08
AI Technical Summary
The existing remote sensing image road extraction method based on deep learning is high in calculation cost, high model complexity, and it is difficult to effectively aggregate multi-scale features, resulting in poor road connectivity and loss of details.
The cascaded attention-guided context aggregation network (AGCANet) is used, and only downsampled in the encoder is downsampled once. The smooth expansion convolutional residual module is used to enhance the feature extraction capability, and multi-level features are fused through the attention-guided feature fusion module (AGFF), and the network is cascaded twice to optimize the road topology.
While reducing the computational complexity, it improves the accuracy and connectivity of road extraction, reduces fragmentation, adapts to complex environments, and performs better than existing methods.
Smart Images
Figure CN120451768A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of machine learning, and in particular to the fields of remote sensing image processing, deep learning, and machine learning. Background Art
[0002] The methods for automatically extracting road information from remote sensing images based on deep learning can be mainly divided into:
[0003] 1) Road extraction methods based on Visual Transformers (ViTs); these methods can effectively model long-range context in images and have achieved good results in natural image processing. However, they are computationally expensive and, without strong regularization, often underperform convolutional neural networks (CNNs) of the same size on medium-sized datasets.
[0004] 2) Graph-based architectures: To better maintain road connectivity, some methods use graph vertices and edges to represent road intersections and curved structures. While graph-based representations improve road connectivity, many methods still incur high training costs due to the oversampling strategy often used when defining graph vertices in the original annotations.
[0005] 3) Convolutional neural network-based encoder-decoder architecture: Given limited data, CNNs have better generalization capabilities than ViTs in road extraction tasks due to their translation equivariance and locality. Current CNN-based road extraction methods typically include complex multi-scale feature fusion modules, which introduce a large number of parameters and limit their feasibility in real-world applications. At the same time, downsampling shallow feature maps results in the loss of key details of long and narrow roads, thereby degrading road connectivity, while upsampling deep feature maps may introduce inaccurate semantic information. Therefore, these methods struggle to effectively aggregate multi-scale features, and the fused features cannot provide the decoder with accurate road structure. Summary of the Invention
[0006] In order to overcome the shortcomings of the existing technology, the present invention provides a remote sensing image road extraction method based on a cascaded attention-guided context aggregation network.
[0007] In order to alleviate the problems of high computational cost and high model complexity in the existing technology, the present invention provides a method for road extraction from remote sensing images based on a cascaded attention-guided context aggregation network. The present invention adopts a CNN-based encoder-decoder architecture (attention-guided context aggregation network, AGCANet). The original image is downsampled only once in the encoder to retain the detailed information of the road as much as possible. At the same time, in order to improve efficiency, multiple smoothed dilated convolution residual modules are used to replace the convolution residual modules in the traditional CNN to obtain a larger receptive field while reducing computational complexity. An attention-guided feature fusion module (AGFF) is designed between the encoder and decoder to fuse multi-level context features. AGCANet is cascaded twice (W-AGCANet) to optimize the road topology and enhance the model representation capability. Overall, the proposed method can perform parallel block reasoning on large-area remote sensing images, enhance the model representation capability on the basis of reducing the algorithm complexity, and can accurately and efficiently extract road information.
[0008] The specific steps of the technical solution adopted by the present invention to solve its technical problem are:
[0009] The present invention comprises the following steps:
[0010] Step 1: Design a road segmentation network using an encoder-decoder architecture. The road segmentation network is an attention-guided context aggregation network (AGCANet), which consists of an encoder, an attention-guided feature fusion module (AGFF), and a decoder.
[0011] Step 2: The encoder consists of three 3×3 convolutional layers and six cascaded smoothed dilated convolution residual modules;
[0012] The original image is initially encoded using three 3×3 convolutional layers, and the encoded feature map is then fed into six cascaded smoothed dilated convolution residual modules to further enhance the model’s road feature extraction capabilities.
[0013] Step 3: After the image is encoded by the encoder, it is input into the AGFF module and the fused feature map F is output. embed , used to fuse features at different levels to improve the model's ability to extract multi-scale features of images;
[0014] Step 4: The AGFF module outputs the fused feature map F embed After that, the input is gradually upsampled in the decoder to generate a road segmentation result map with the same size as the input image;
[0015] Step 5: The encoder, AGFF, and decoder are cascaded to form the road map extraction network AGCANet. Using a two-stage architecture, two identical AGCANets are cascaded twice to form a W-AGCANet network. Specifically, the road segmentation output of the first AGCANet is concatenated with the original input image along the channel dimension and used as the input of the second AGCANet. The road segmentation output of the second AGCANet becomes the final predicted road map.
[0016] Step 6: Train the W-AGCANet network and optimize the model by setting the loss function and learning parameters to obtain the final road extraction results.
[0017] Furthermore, the specific steps of step 2 are as follows: in the encoder, considering that the traditional layered encoder-decoder network reduces the resolution and doubles the number of channels at each stage, which will significantly degrade the quality of road features, the original image is downsampled once to preserve the road details as much as possible. Therefore, the strides of the three 3×3 convolutional layers are set to (1, 1, 2) respectively;
[0018] Furthermore, the specific steps of step 2 are as follows: considering the narrowness, connectivity, complexity and slender distribution of roads in remote sensing images, it is crucial to expand the receptive field of feature points without losing detail information. The smooth dilated convolution residual module is two cascaded smooth dilated convolution layers with jump connections, each smooth dilated convolution layer includes a segmentation shared convolution layer and a dilated convolution layer, and each convolution layer is followed by instancenorm normalization and relu activation function for feature normalization and nonlinear processing; wherein the segmentation shared convolution layer is a convolution layer with a kernel size of 1×1 and a number of input and output channels of 1, and the dilation rate of the dilated convolution layer is (3, 3, 3, 7, 7, 7); 6 stacked smooth dilated convolution residual modules are used to enhance the model's road feature representation ability and expand the receptive field of feature points, while reducing the impact of grid artifacts on road segmentation.
[0019] Furthermore, the specific steps of step 3 are as follows: the overall architecture of the AGFF module is as follows Figure 2 As shown, it includes a convolutional layer σ with a kernel size of 3×3, and the input of σ is the feature map F output at different stages of the encoder l 、F m 、F h , the output is the corresponding spatial attention map W l 、W m 、W h ; Specifically, F l is the feature map output by the third 3×3 convolutional layer in step 2, F m 、Fh They are the output feature maps of the third and last smooth dilated convolution residual modules in step 2 respectively; finally, the spatial attention maps W at different stages are l 、W m 、W h Respectively with the corresponding feature map F l 、F m 、F h After multiplication, linear combination is performed to obtain the fused feature map F embed :
[0020] (W l ,W m ,W h )=σ(concat(F l ,F m ,F h )) (1)
[0021]
[0022] Where concat is the feature map F l 、F m 、F h Connect from the channel dimension, σ is the convolution layer with a kernel size of 3×3, is the Hadamard product of the matrix, F embed The AGFF module outputs the feature map. The AGFF module calculates the positional correlation between feature maps at different stages through network training, that is, the spatial and channel attention maps of different feature layers, so as to aggregate features at different levels from the perspective of channels and space.
[0023] Furthermore, the specific steps of step 4 are as follows: the input of the decoder is the feature map F obtained in step 3 embed , after upsampling by a decoder consisting of three transposed convolutional layers with strides of (1, 1, 2), the output is a rough road map with the same resolution as the original image.
[0024] Furthermore, the specific steps of step 5 are as follows: the encoder, AGFF, and decoder described in steps 2 to 4 are cascaded to form an AGCANet, and the AGCANet is cascaded twice to form a W-type network, namely W-AGCANet; the specific steps are as follows:
[0025] The rough road map output by the first AGCANet and the original image are connected in the channel dimension and input into the second AGCANet, as shown in Equation (3):
[0026]
[0027] Among them, φ is the overall network W-AGCANet, For the codec network AGCANet described in steps 1 to 4, For the first AGCANet, is the second AGCANet, and x is the input remote sensing image. The network adopts a cascade architecture to efficiently and accurately predict and correct road locations in an iterative manner.
[0028] Furthermore, the specific method of step 6 is: the loss function of network training is shown in formula (4):
[0029]
[0030] Among them, l is the sum of binary cross entropy loss (BCE Loss) and Dice Loss, φ(x) is the output of the entire network, and y is the labeled road map. is the rough road map output from the first stage, is the predicted road map of the second stage, which is the final output, i.e., and Figure 3 and Figure 4 Predicted road graphs using different algorithms on two large datasets.
[0031] An electronic device comprises: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the method described above.
[0032] A computer-readable storage medium stores program code, which can be called by a processor to execute the method described above.
[0033] The beneficial effects of the present invention are:
[0034] (1) In terms of visualization, a comprehensive evaluation was conducted on two large-scale public datasets. The results showed that the proposed method produces smoother and more coherent road detection results than all the comparison methods, and performs well in maintaining the integrity of road topology, reducing fragmentation, and adapting to complex environments.
[0035] (2) In terms of quantitative analysis, a comprehensive evaluation was conducted on two large-scale public datasets. The results showed that the proposed algorithm outperformed existing algorithms in terms of detection accuracy, recall rate, F1 score, and APSL (average path length similarity, an indicator for measuring road connectivity).
[0036] (3) The W-AGCANet proposed in this paper performs better than most state-of-the-art methods in terms of FLOPs and parameter count, and has lower computational complexity. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] Figure 1 The overall framework of the proposed method.
[0038] Figure 2 This is the structural diagram of the attention-guided feature fusion module.
[0039] Figure 3 This is the effect of the algorithm proposed in this paper on the Deepglobe dataset.
[0040] Figure 4 This is the effect of the algorithm proposed in this invention on the Massachusetts dataset. DETAILED DESCRIPTION
[0041] The present invention will be further described below with reference to the accompanying drawings and examples.
[0042] Figure 1 This is a flow chart of a method for extracting roads from remote sensing images based on a cascaded attention-guided context aggregation network provided by the present invention. Figure 1 As shown, the method implementation steps include:
[0043] Step 1: Design a road segmentation network using an encoder-decoder architecture, specifically an attention-guided context aggregation network (AGCANet), which includes an encoder, an attention-guided feature fusion module (AGFF), and a decoder.
[0044] Step 2: The encoder consists of three 3×3 convolutional layers and six cascaded dilated convolutional residual modules. The original image is initially encoded using three 3×3 convolutional layers with strides of (1, 1, 2). The encoded feature map is then fed into six cascaded dilated convolutional residual modules, further enhancing the model's ability to extract road features.
[0045] The smoothed dilated convolutional residual module consists of two cascaded smoothed dilated convolutional layers with skip connections. Each smoothed dilated convolutional layer includes a split-shared convolutional layer and a dilated convolutional layer. Each convolutional layer uses instancenorm normalization and ReLU activation function for feature normalization and nonlinear processing. The split-shared convolutional layer is a convolutional layer with 1 input and output channel and a kernel size of (3, 3, 3, 7, 7) × (3, 3, 3, 7, 7, 7); the dilation rate of the dilated convolutional layer is (3, 3, 3, 7, 7, 7).
[0046] Step 3: After the image is encoded by the encoder, it is input into the AGFF module and the fused feature map F is output. embed The overall architecture of this module is as follows Figure 2 As shown, it includes a convolutional layer σ with a kernel size of 3×3, and the input of σ is the feature map F output at different stages of the encoder l 、F m 、F h , the output is the corresponding spatial attention map W l 、W m 、W h . Specifically, F l is the feature map output by the third 3×3 convolutional layer in step 2, F m 、F h They are the output feature maps of the third and last smooth dilated convolution residual modules in step 2 respectively. Finally, the spatial attention maps W at different stages are l 、W m 、W h Respectively with the corresponding feature map F l 、F m 、F h Linear combination after multiplication:
[0047] (W l ,W m ,W h )=σ(concat(F l ,F m ,F h )) (5)
[0048]
[0049] Where concat is the feature map F l 、F m 、F h Connect from the channel dimension, σ is the convolution layer with a kernel size of 3×3, is the Hadamard product of the matrix, F embed This is the feature map output by the AGFF module. The AGFF module calculates the positional correlation between feature maps at different stages through network training, namely the spatial and channel attention maps of different feature layers, allowing it to aggregate features at different levels from a channel and spatial perspective.
[0050] Step 4: AGFF module outputs the fused feature map F embed After that, it is upsampled by three transposed convolutional layers with stride sizes of (1, 1, 2) to output a rough road map with the same resolution as the original image.
[0051] Step 5: Combine the encoder, AGFF, and decoder in a cascade to form a road map extraction network AGCANet; then use a two-stage architecture, that is, cascade two identical AGCANets twice to form a W-type network W-AGCANet. Specifically, the road segmentation result map output by the first AGCANet and the original input image are connected in the channel dimension and used as the input of the second AGCANet. The road segmentation result map output by the second AGCANet is the final predicted road map. The encoder, AGFF, and decoder described in steps 2 to 4 above are cascaded to form AGCANet. Cascading AGCANet twice forms a W-type network, namely W-AGCANet. Specifically, the rough road map output by the first AGCANet and the original image are connected in the channel dimension and input into the second AGCANet, as shown in formula (7):
[0052]
[0053] Among them, φ is the overall network W-AGCANet, For the codec network AGCANet described in steps 1 to 4, For the first AGCANet, is the second AGCANet, and x is the input remote sensing image. The network adopts a cascade architecture to efficiently and accurately predict and correct road locations in an iterative manner.
[0054] Step 6: Train the W-AGCANet network. By setting the loss function and learning parameters, the model is trained and optimized to obtain the final road extraction result. The loss function of the network training is shown in formula (8):
[0055]
[0056] Among them, l is the sum of binary cross entropy loss (BCE Loss) and Dice Loss, φ(x) is the output of the entire network, and y is the annotated road map. is the rough road map output from the first stage, The predicted road map of the second stage, i.e. the final output (i.e. the and ). Figure 3 and Figure 4 Predicted road graphs using different algorithms on two large datasets.
[0057] Figure 3 and Figure 4The input images, true labels, existing state-of-the-art algorithms, and predicted road maps of the proposed algorithm in DeepGlobe and Massachusetts road datasets are shown. It can be seen that the proposed algorithm outperforms other methods in adapting to different scenarios and can better detect road edges and structural features in both datasets. This shows that the proposed algorithm has the ability to extract road features in diverse scenarios, and arrows are used to specifically mark challenging areas. For example, in Figure 3 In the figure, several roads are not marked in the ground-truth road map (the first and third rows). All algorithms predict some of these missing roads, but the algorithm proposed in this paper performs best in maintaining topological integrity, highlighting its superior ability to preserve road topological continuity (yellow arrows). Furthermore, the road masks predicted by other methods exhibit varying degrees of fragmentation, while the algorithm proposed in this paper better preserves road connectivity (red arrows).
[0058] exist Figure 4 In the first row, a two-way highway is incorrectly marked as a single lane. All algorithms correctly extract the two-way highway, but the algorithm proposed in this paper performs better in extracting local detailed information (such as roundabouts and roads blocked by surrounding trees or shadows). For more complex scenes, such as Figure 4 In the fourth row, the prediction results of the algorithm proposed in this paper are closer to the true label, while other methods tend to identify the road as the background and generate fragmented road maps.
[0059] Overall, the proposed algorithm accurately predicts road networks in challenging environments, such as those with multiple blocks and intersections in dense urban areas, irregularly curved roads, overpasses, and multi-lane highways. Furthermore, the predicted masks are sharp and clear, accurately reflecting the road geometry.
[0060] Table 1 shows the performance comparison between the proposed method and other methods on the Massachusetts dataset.
[0061] Table 1 Performance comparison on the Massachusetts dataset
[0062] method Accuracy Recall F1 score clDice APLS Casnet 0.740 0.779 0.759 0.851 0.697 D-Linknet 0.775 0.727 0.751 0.849 0.636 SGCN 0.747 0.758 0.752 0.852 0.655 CoANet 0.755 0.739 0.747 0.840 0.659 RCFSNet 0.782 0.755 0.768 0.862 0.701 WNet 0.770 0.702 0.734 0.828 0.607 The present invention 0.817 0.791 0.804 0.904 0.793
[0063] All metrics range from 0 to 1, with larger values indicating better results. clDice is the Dice parameter value based on road centerlines (the Dice coefficient is a set similarity metric commonly used to calculate the similarity between two samples), and APLS is the average path length similarity. Precision indicates the proportion of pixels predicted by the algorithm to be road pixels; recall indicates the algorithm's ability to correctly identify all road pixels; the F1 score comprehensively measures the algorithm's precision and recall, serving as the harmonic mean of the two. clDice measures the completeness and continuity of road extraction; and APLS comprehensively assesses the topology of the road network and is suitable for evaluating the connectivity and accuracy of long paths. The underlined results represent the second-best algorithm, while the bolded results represent the best.
[0064] Table 2 shows the performance comparison of the proposed method and other methods on the Deepglobe dataset.
[0065] Table 2 Performance comparison on Deepglobe dataset
[0066] method Accuracy Recall F1 score clDice APLS Casnet 0.604 0.767 0.676 0.428 0.505 D-Linknet 0.697 0.696 0.696 0.538 0.557 SGCN 0.636 0.721 0.676 0.442 0.509 CoANet 0.700 0.624 0.660 0.567 0.499 RCFSNet 0.762 0.660 0.708 0.600 0.602 WNet 0.680 0.574 0.623 0.522 0.389 The present invention 0.813 0.816 0.815 0.753 0.739
[0067] Table 3 compares the computational complexity of the method proposed in this invention with other methods.
[0068] Table 3 Comparison of computational complexity
[0069] Evaluation Metrics Casnet D-Linknet SGCN CoANet RCFSNet The present invention Number of floating-point operations 9.78G 6.67G 61.9G 69.3G 45.6G 30.52G Parameter quantity 3.83M 31.1M 42.7M 59.1M 76.7M 1.4M
[0070] The lower the number of floating-point operations, the smaller the amount of calculation. The number of parameters represents the size of the model, and the smaller the value, the simpler the model. The algorithm proposed in this invention has a lower number of floating-point operations than most algorithms and has the least number of parameters.
Claims
1. A remote sensing image road extraction method based on cascaded attention-guided context aggregation network, characterized by The steps include: Step 1: Design a road segmentation network using an encoder-decoder architecture. The road segmentation network is an attention-guided context aggregation network consisting of an encoder, an attention-guided feature fusion module, and a decoder. Step 2: The encoder consists of three 3×3 convolutional layers and six cascaded smoothed dilated convolution residual modules; The original image is initially encoded using three 3×3 convolutional layers, and the encoded feature map is then fed into six cascaded smoothed dilated convolution residual modules to further enhance the model’s road feature extraction capabilities. Step 3: After the image is encoded by the encoder, it is input into the AGFF module and the fused feature map F is output. embed , used to fuse features at different levels to improve the model's ability to extract multi-scale features of images; Step 4: AGFF module outputs the fused feature map F embed After that, the input is gradually upsampled in the decoder to generate a road segmentation result map with the same size as the input image; Step 5: The encoder, AGFF, and decoder are cascaded to form the road map extraction network AGCANet. Using a two-stage architecture, two identical AGCANets are cascaded twice to form a W-AGCANet network. Specifically, the road segmentation output of the first AGCANet is concatenated with the original input image along the channel dimension and used as the input of the second AGCANet. The road segmentation output of the second AGCANet becomes the final predicted road map. Step 6: Train the W-AGCANet network and optimize the model by setting the loss function and learning parameters to obtain the final road extraction results.
2. The method for extracting roads from remote sensing images using a cascaded attention-guided context aggregation network according to claim 1, characterized in that: The specific steps of step 2 are as follows: in the encoder, set the step sizes of the three 3×3 convolutional layers to (1, 1, 2) respectively.
3. The method for extracting roads from remote sensing images using a cascaded attention-guided context aggregation network according to claim 1, characterized in that: The specific steps of step 2 are as follows: the smoothed dilated convolution residual module is two cascaded smoothed dilated convolution layers with jump connections, each smoothed dilated convolution layer includes a segmentation shared convolution layer and a dilated convolution layer, and each convolution layer is followed by instancenorm normalization and relu activation function for feature normalization and nonlinear processing; the segmentation shared convolution layer is a convolution layer with 1 input and output channels and a kernel size of 1×1, and the dilation rate of the dilated convolution layer is (3, 3, 3, 7, 7, 7); 6 stacked smoothed dilated convolution residual modules are used to enhance the model's road feature representation ability and expand the receptive field of feature points, while reducing the impact of grid artifacts on the road segmentation effect.
4. The method for extracting roads from remote sensing images using a cascaded attention-guided context aggregation network according to claim 1, characterized in that: The specific steps of step 3 are as follows: the AGFF module includes a convolutional layer σ with a kernel size of 3×3, and the input of σ is the feature map F output at different stages of the encoder l 、F m 、F h , the output is the corresponding spatial attention map W l 、W m 、W h ; Specifically, F l is the feature map output by the third 3×3 convolutional layer in step 2, F m 、F h They are the output feature maps of the third and last smooth dilated convolution residual modules in step 2 respectively; finally, the spatial attention maps W at different stages are l 、W m 、W h Respectively with the corresponding feature map F l 、F m 、F h After multiplication, linear combination is performed to obtain the fused feature map F embed : (W l ,W m ,W h )=σ(concat(F l ,F m ,F h )) (1) Where concat is the feature map F l 、F m 、F h Connect from the channel dimension, σ is the convolution layer with a kernel size of 3×3, is the Hadamard product of the matrix, F embed The AGFF module outputs the feature map. The AGFF module calculates the positional correlation between feature maps at different stages through network training, that is, the spatial and channel attention maps of different feature layers, so as to aggregate features at different levels from the perspective of channels and space.
5. The method for extracting roads from remote sensing images using a cascaded attention-guided context aggregation network according to claim 1, characterized in that: The specific steps of step 4 are: the input of the decoder is the feature map F obtained in step 3 embed , after upsampling by a decoder consisting of three transposed convolutional layers with strides of (1, 1, 2), the output is a rough road map with the same resolution as the original image.
6. The method for extracting roads from remote sensing images using a cascaded attention-guided context aggregation network according to claim 1, characterized in that: The specific steps of step 5 are as follows: the encoder, AGFF, and decoder described in steps 2 to 4 are cascaded to form an AGCANet, and the AGCANet is cascaded twice to form a W-type network, namely W-AGCANet; the specific steps are as follows: The rough road map output by the first AGCANet and the original image are connected in the channel dimension and input into the second AGCANet, as shown in Equation (3): Among them, φ is the overall network W-AGCANet, For the codec network AGCANet described in steps 1 to 4, For the first AGCANet, is the second AGCANet, x is the input remote sensing image, and the entire network adopts a cascade architecture to efficiently and accurately predict and correct road positions in an iterative manner.
7. The method for extracting roads from remote sensing images using a cascaded attention-guided context aggregation network according to claim 1, characterized in that: The specific method of step 6 is: the loss function of network training is shown in formula (4): Among them, l is the sum of binary cross entropy loss (BCE Loss) and Dice Loss, φ(x) is the output of the entire network, and y is the labeled road map. is the rough road map output from the first stage, This is the predicted road map of the second stage, which is the final output.
8. An electronic device, characterized in that: include: one or more processors; Memory; One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to perform the method according to any one of claims 1 to 7.
9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores program code, and the program code can be called by a processor to execute the method according to any one of claims 1 to 7.