Remote sensing image road extraction method based on double encoders and frequency perception

Abstract

The invention belongs to the technical field of remote sensing image processing and computer vision, and particularly relates to a remote sensing image road extraction method based on double encoders and frequency perception. The method mainly comprises the following steps: 1) constructing a double-encoder network architecture, integrating a pre-trained SAM image encoder and a D-LinkNet backbone network, respectively extracting global semantic features and local geometric features, and enhancing the cross-domain generalization ability of a model through feature alignment and fusion; 2) designing a multi-direction topology perception aggregation module, and effectively recovering the road connectivity of the sheltered area through strip convolution explicit coding road topology prior; and 3) introducing an adaptive frequency sensing fusion module, and adopting a content adaptive low-pass and high-pass filtering mechanism to optimize boundary definition and suppress intra-class inconsistency. According to the method, three challenges of insufficient generalization, topological fracture and boundary blur in high-resolution remote sensing image road extraction are solved, and a reliable scheme is provided for accurate road network extraction in a complex city scene.

Description

Technical Field

[0001] This invention belongs to the field of remote sensing image processing and computer vision technology, specifically relating to a method for road extraction from remote sensing images based on dual encoders and frequency sensing. Background Technology

[0002] Road extraction, as an important research direction in image processing, has wide-ranging applications in remote sensing image analysis, involving several key areas such as map creation, traffic rescue, and vehicle navigation. For example, in map creation, efficient road extraction technology can significantly reduce the input of human resources; in emergency rescue scenarios, quickly and accurately identifying available roads is crucial to improving rescue efficiency; at the same time, improving the accuracy of road extraction also helps to enhance the reliability of vehicle navigation systems.

[0003] Traditional methods for road extraction from remote sensing images are primarily based on Convolutional Neural Networks (CNNs) and their variants, constructing network models through stacked convolutions and pooling operations. However, these methods have certain limitations: First, traditional models often neglect long-range dependencies between road structures. Although deep convolutions can expand the receptive field, the number of model parameters and computational complexity increase dramatically with network depth, leading to decreased efficiency. Second, during feature extraction, convolutional operations give equal attention to features of roads and non-road areas (such as buildings and vegetation). However, in actual remote sensing images, road areas typically constitute a small proportion and are easily overwhelmed by background information during training, affecting the model's ability to discriminate road features. Furthermore, remote sensing images are affected by factors such as geographical location and imaging conditions, resulting in significant differences in their spectral, textural, and geometric features, further increasing the difficulty of road extraction. Therefore, effectively enhancing the model's attention mechanism for road features and improving its ability to perceive global contextual information has become a key issue in improving the accuracy of road extraction from remote sensing images.

[0004] With the widespread adoption of high-resolution remote sensing imagery, road extraction technology has evolved from manual intervention to automation and intelligent processing. Early methods relied primarily on professionals manually interpreting and interactively labeling images based on prior knowledge. While accurate and reliable, these methods were inefficient and demanded high levels of expertise from operators, making them unsuitable for processing massive amounts of data. In recent years, deep learning methods, especially convolutional neural networks (CNNs), have demonstrated significant advantages in automatic image feature extraction, gradually becoming the mainstream technology for road extraction. Furthermore, the successful application of the Transformer architecture and its variants in computer vision has provided new insights into road extraction. These methods capture global features through self-attention mechanisms, effectively addressing the challenge of identifying occluded roads or roads that are similar to the background, further enhancing road extraction performance.

[0005] Currently, existing road extraction methods mainly include the following categories:

[0006] 1) Traditional methods based on human-designed features: Before the widespread adoption of deep learning technology, road extraction mainly relied on manually designed features and rules, including classification-based algorithms, knowledge-driven reasoning methods, mathematical morphology processing, active contour models, and dynamic programming algorithms. While these methods can improve the accuracy and efficiency of extraction in certain scenarios, their performance is highly dependent on human experience. They require adjusting parameters or designing specific rules for different image characteristics, have poor generalization ability, and are difficult to automate on a large scale.

[0007] 2) Semantic segmentation methods based on convolutional neural networks: Deep learning models such as Fully Convolutional Networks (FCN), U-Net, D-LinkNet, DeepLab series, and HRNetV2 achieve end-to-end road segmentation through multi-scale feature extraction and encoder-decoder structures. These methods can automatically learn road features, significantly improving processing efficiency. However, due to their relatively fixed model structure, they require a large amount of labeled data for training and have high requirements for the representativeness and quality of the dataset, resulting in significant initial data labeling costs.

[0008] 3) Global feature extraction methods based on Transformers: Models such as RADANet and Pyramid Deformable Vision Transformer utilize self-attention mechanisms to model the global contextual information of roads, effectively capturing long-distance dependencies and improving road recognition performance in complex scenarios. However, Transformer-type models typically have high computational complexity, large parameter scale, and strict hardware resource requirements, making them difficult to deploy in lightweight or real-time application scenarios.

[0009] In summary, existing road extraction methods still face the following challenges in terms of practicality, generalization ability, and efficiency: traditional methods rely on manual priors and lack adaptability; CNN-based methods require a large amount of labeled data, resulting in high costs; while Transformer-type methods have the advantage of global modeling, they consume a lot of computational resources and cannot meet the requirements for lightweight design.

[0010] Therefore, there is an urgent need to develop a new network architecture that can take into account multi-scale feature fusion, road linear structure characteristic modeling, and computational efficiency, so as to improve the universality, accuracy and practicality of road extraction from remote sensing images. Summary of the Invention

[0011] This invention aims to address the problems of topological breaks, blurred boundaries, and insufficient generalization ability in existing road extraction methods under complex scenarios. It proposes a remote sensing image road extraction method based on dual encoders and frequency sensing. This method achieves high-precision and high-connectivity remote sensing image road extraction through dual-path feature encoding, multi-directional topological sensing aggregation, and adaptive frequency sensing fusion.

[0012] To achieve the above-mentioned technical objectives and effects, the present invention is implemented through the following technical solution:

[0013] This invention provides a method for road extraction from remote sensing images based on dual encoders and frequency sensing, comprising the following steps:

[0014] S1. Construct a road extraction dataset, including data acquisition, preprocessing, enhancement, and segmentation;

[0015] S2. Design a dual encoder architecture that integrates the SAM image encoder and the D-LinkNet backbone network;

[0016] S3. Design a multi-directional topology sensing aggregation module to enhance road connectivity through multi-scale four-directional strip convolution;

[0017] S4. Design an adaptive frequency sensing fusion module to optimize boundary accuracy through content-adaptive high-pass and low-pass filtering;

[0018] S5. Construct a multi-scale enhanced decoder, integrating the modules from steps S3 and S4 for feature decoding;

[0019] S6. Model training and optimization, using a hybrid loss function and learning rate decay strategy;

[0020] S7. Model performance evaluation, including quantitative indicator analysis and qualitative visualization verification.

[0021] Furthermore, high-resolution remote sensing images were acquired, cropped into 512×512 pixel RGB images, and road pixels were manually labeled; data augmentation was performed using random rotation, horizontal flipping, and vertical flipping; the dataset was divided into training, validation, and test sets according to the proportions, and corresponding file lists and metadata files were generated.

[0022] Furthermore, the dual-encoder architecture comprises a SAM image encoder and a D-LinkNet encoder operating in parallel. The SAM image encoder, based on a pre-trained ViT architecture, extracts global semantic features and freezes parameters to maintain generalization ability. The D-LinkNet encoder, based on ResNet-34, uses dilated convolutions to expand the receptive field and extract local geometric features. The dual-encoder features are aligned in spatial dimensions through projection and downsampling, and fused through element-wise addition and 1×1 convolutions, as expressed in the formula:

[0023] ;

[0024] in, Features of D-LinkNet Features of SAM This is a projection operation.

[0025] Furthermore, the multi-directional topology-aware aggregation module captures the linear topology of roads through multi-scale strip convolution, using dilated strip convolutions in four directions (horizontal, vertical, and two diagonals) at three scales (kernel sizes 5, 9, and 13). The diagonal convolutions are converted to horizontal convolutions through coordinate transformation, achieving efficient multi-directional feature extraction. Multi-scale features are fused through channel stitching and 1×1 convolutions to enhance road connectivity under occlusion, as expressed by the formula:

[0026] .

[0027] Furthermore, the adaptive frequency-aware fusion module separates high- and low-frequency information through content adaptive filtering; the adaptive low-pass generator generates a low-pass filter kernel through the content encoder to smooth internal road features and reduce intra-class inconsistencies; the adaptive high-pass generator generates a high-pass filter kernel to enhance boundary details and suppress background noise; the fusion process is expressed by the following formula:

[0028] ;

[0029] in, This is a Qualcomm characteristic. It is a low-pass characteristic. These are the original high-frequency features.

[0030] Furthermore, the multi-scale enhancement decoder progressively upsamples and fuses encoder features: each stage of the decoder integrates AFAF and MTAA modules to recover spatial details through skip connections; the AFAF module fuses high and low resolution features in each layer of the decoder to optimize boundary localization; the MTAA module enhances the continuity of road topology and reduces breaks caused by occlusion; the final output is upsampled to the original resolution through transposed convolution to generate a road probability map.

[0031] Furthermore, the model training employs a hybrid loss function:

[0032] ;

[0033] For binary cross-entropy loss, The loss is Dice; the optimizer uses Adam with an initial learning rate of 2×10⁻⁶. -4 A step size decay strategy is adopted; the batch size is 8, and the training cycle is stopped early based on the performance of the validation set.

[0034] Furthermore, the model evaluation metrics include intersection-over-union ratio, F1 score, precision, recall, and overall accuracy; comparative experiments and ablation studies are conducted on the dataset constructed in step S1 to verify the model's advantages in road connectivity and boundary accuracy in occluded scenarios.

[0035] To address the common problems of broken thin roads, blurred boundaries, and insufficient cross-domain generalization ability in complex scenarios, existing road extraction methods in this invention provide a road extraction network (FADENet) based on dual encoders and frequency-aware fusion. This method, through a dual-path encoder design, deeply fuses the robust global prior features of a general visual foundation model (such as SAM) with multi-scale local geometric features extracted by the task backbone network (such as ResNet), effectively enhancing the model's adaptability to complex scenarios and its cross-domain generalization ability. Considering the unique elongated and continuous topological structure of road targets, and the tendency to lose high-frequency edge information during downsampling, this invention innovatively proposes a frequency-aware fusion module (AFAF). During the decoding process, it adaptively weights and fuses high-resolution features (rich in high-frequency edge details) and deep low-resolution features (rich in low-frequency semantic information) from the encoded path, significantly improving boundary localization accuracy and suppressing intra-class inconsistencies. Meanwhile, a multi-directional topology-aware aggregation module (MTAA) is introduced into the decoder. This module explicitly models the linear features of roads through a multi-scale star-shaped convolutional structure, effectively mitigating road fragmentation caused by occlusion and significantly enhancing the topological continuity of the extracted results. Furthermore, a multi-scale dilated convolutional module (Dblock) expands the receptive field to aggregate the global context, and a composite loss function combined with test-time augmentation (TTA) further optimizes the model's training stability and inference robustness. In summary, this invention fully leverages the complementary advantages of dual encoders, frequency-aware detail preservation, and topology-aware structural modeling to achieve high-precision, high-connectivity extraction of road networks from high-resolution remote sensing imagery. While maintaining high efficiency, it significantly improves practicality and reliability in complex scenarios. This method supports road extraction from large-scale high-resolution remote sensing imagery, and its topological integrity and boundary clarity in complex urban environments are verified through multi-scale visualization.

[0036] The beneficial effects of this invention are:

[0037] 1. It achieves stable performance in complex urban scenes by fusing the robust global prior provided by the SAM image encoder with the multi-scale local semantics extracted by the ResNet backbone in high-resolution remote sensing images, effectively overcoming the domain adaptation problem caused by differences in different regions and imaging conditions.

[0038] 2. Adaptive allocation and fusion of high-frequency components of thin-line road edges and low-frequency components of regional semantics are performed, taking into account both edge clarity and connectivity. The innovative multi-directional topology perception aggregation module explicitly models the linear structural prior of the road, and effectively establishes long-distance spatial dependencies through multi-scale, multi-directional strip convolution, significantly reducing road breakage caused by occlusion.

[0039] 3. The overall network structure is reasonably designed. While introducing powerful functional modules, the model achieves a good balance between accuracy and computational efficiency through lightweight decoding design and modular integration.

[0040] 4. The systematic experimental evaluation fully verified the advanced nature and practicality of the method of this invention from multiple dimensions, including quantitative indicators, qualitative effects and module effectiveness.

[0041] Of course, any product implementing this invention does not necessarily need to achieve all of the above advantages at the same time. Attached Figure Description

[0042] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1 This is a flowchart of an embodiment of the present invention;

[0044] Figure 2 This is a model framework diagram of an embodiment of the present invention;

[0045] Figure 3 This is a structural diagram of the adaptive frequency sensing fusion module according to an embodiment of the present invention;

[0046] Figure 4 This is a structural diagram of the multi-directional topology sensing aggregation module according to an embodiment of the present invention;

[0047] Figure 5 This is a diagram illustrating the qualitative experimental results of an embodiment of the present invention. Detailed Implementation

[0048] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0049] This invention provides a method for road extraction from remote sensing images based on dual encoders and frequency sensing, comprising the following steps:

[0050] Step S1: Input and Preprocessing

[0051] Read high-resolution remote sensing images and perform preprocessing such as normalization; to adapt to the subsequent dual encoder branches, scale the images used to prompt the encoder branches to a fixed scale (such as 1024×1024).

[0052] Step S2: Feature Extraction from Dual Encoders

[0053] A dual-branch encoder structure is used to extract features in parallel:

[0054] The main semantic encoder branch uses a pre-trained convolutional neural network (such as ResNet-34) as its backbone to extract multi-scale feature maps (e1, e2, e3, e4). Its output feature map dimensions can be expressed as:

[0055]

[0056]

[0057] Hint: Image encoder branch that incorporates the Segment Anything Model (SAM) (such as ViT-H or the lightweight MobileSAM) branch. The parameters of this branch are frozen during training to maintain its strong generalization ability. The preprocessed image (scaled to 1024×1024) is fed into the SAM encoder to obtain global prior features. Subsequently, channel projection is performed using 1×1 convolution, batch normalization (BN), and ReLU activation function to align it with the high-level semantic features of the main encoder. The number of channels is the same:

[0058]

[0059] Finally, the two are fused element-wise to obtain the enhanced fusion feature. :

[0060]

[0061] Step S3: Context-Aware Enhancement

[0062] Fusion features Input a multi-scale dilated convolutional module (Dblock). This module contains multiple parallel dilated convolutional layers with different dilation rates (e.g., 1, 2, 4, 8), and its operation is defined as follows:

[0063]

[0064] Without reducing the resolution of the feature map, the receptive field is significantly expanded, multi-scale contextual information is effectively aggregated, and the model's ability to perceive the global road layout and background interference is enhanced.

[0065] Step S4: Frequency-aware cross-layer fusion and step-by-step decoding

[0066] A frequency-aware fusion module is proposed, introducing adaptive frequency-aware fusion at multiple levels of the decoding path (e.g., 1 / 16, 1 / 8, 1 / 4 scales). The core of this module lies in processing smooth regions (low-frequency semantics) and edge details (high-frequency information) respectively through content-adaptive low-pass and high-pass filtering. Specifically, it processes deep, upsampled low-resolution strong semantic features. Adaptive low-pass filtering is applied to reduce intra-class inconsistencies within road areas:

[0067]

[0068] in Dynamically generated by the content encoder. High-resolution features from the shallow layer. Adaptive high-pass filtering is performed to enhance boundary details.

[0069]

[0070] Finally, the processed high- and low-frequency features are fused:

[0071]

[0072] A topology-aware decoding block is proposed, into which the fused features are fed. This decoding block first uses 1×1 convolutions to reduce the channel dimension:

[0073]

[0074] Then, multiple star-shaped convolutional structures of different scales (e.g., kernel sizes of 5, 9, and 13) are used in parallel for multi-directional topology-aware aggregation. Taking horizontal convolution as an example:

[0075]

[0076] This structure explicitly models the linear topological properties of roads through strip convolutions in the horizontal, vertical, and diagonal directions, effectively mitigating the breakage problem caused by occlusion. After multi-scale feature aggregation, it is fused through 1×1 convolutions and then residually connected to the main branch.

[0077]

[0078] Finally, the image is upsampled to a higher resolution using deconvolution. This process is performed step by step to gradually restore the fine spatial structure of the road.

[0079] Step S5: Predict the output

[0080] The high-resolution feature map obtained from the final decoding is upsampled to the original input size through transposed convolution, and then passed through a convolutional layer and a sigmoid activation function, as shown in the following formula:

[0081]

[0082] Where σ is the Sigmoid function. A binary road mask is generated by thresholding (e.g., to 0.5).

[0083]

[0084] Output a probability map where each pixel represents a road. A binarized road mask can be obtained by setting a threshold (e.g., 0.5).

[0085] Step S6: Training and Reasoning

[0086] Supervision is performed using a composite loss function, which is defined as a weighted sum of the binary cross-entropy loss and the Dice loss:

[0087]

[0088] Among them, the binary cross-entropy loss

[0089]

[0090]

[0091] This combination effectively balances pixel-level accuracy and overall region consistency, mitigating the class imbalance problem between roads and backgrounds. The optimizer used is Adam, with an initial learning rate of 2e-4 and a step decay strategy. For data augmentation, random rotations (30°, 60°, 90°, 120°, 150°), horizontal / vertical flipping, and color jitter are employed to improve the model's generalization ability. During training, the parameters of the main encoder and decoder are updated, while the parameters of the SAM image encoder remain frozen.

[0092] For large input images, the inference stage can employ an overlapping sliding window approach for prediction, and the results can be stitched together to reduce boundary effects. Optional test-time enhancements (such as multi-scale / flip averaging) and post-processing operations (such as morphological thinning or skeletonization) can be performed during the inference stage to further improve the connectivity of the road network.

[0093] Step S7: Model performance evaluation and index analysis

[0094] To comprehensively verify the effectiveness, robustness, and superiority of the proposed FADENet model, the inventors conducted a systematic experimental evaluation, covering quantitative indicator analysis, qualitative performance comparison, computational efficiency evaluation, and ablation studies. The invention was compared with several state-of-the-art methods on three challenging remote sensing road extraction datasets (HF Road Dataset, DeepGlobe Dataset, and Massachusetts Dataset). Evaluation metrics included pixel-level Intersection over Union (IoU), F1 score, precision, recall, and overall accuracy (OA), which respectively measure the overlap of segmented regions, overall performance, reliability, completeness, and overall classification accuracy.

[0095]

[0096] Experimental results show that this invention reaches or exceeds the existing advanced level in all indicators. Visual comparisons intuitively demonstrate the advantages of this invention in handling complex scenarios (such as building shadows, vegetation cover, narrow side roads, and complex overpasses). As shown in the figure, areas with severe occlusion often exhibit obvious road breaks or blurred boundaries. This invention, benefiting from the robust priors provided by the dual encoders, the explicit modeling of linear structures by the Multi-Directional Topology Aware Aggregation (MTAA) module, and the enhancement of boundary details by the Adaptive Frequency Aware Fusion (AFAF) module, can recover continuous, complete, and clearly defined road networks. Through systematic ablation studies, the contributions of the dual encoder design, the MTAA module, and the AFAF module were quantitatively evaluated. The results show that removing any core module leads to a significant performance degradation, especially in the IoU metric, which measures road connectivity. This fully demonstrates the necessity and effectiveness of the components proposed in this invention in improving the completeness and accuracy of road extraction. Furthermore, the ablation experiments also revealed the advantageous scenarios for different modules: the MTAA module contributes most significantly in urban environments with regular building shadows and significant occlusion; while the AFAF module plays a crucial role in rural vegetation-covered areas with severe spectral confusion and blurred boundaries.

[0097] The specific embodiments of the present invention are as follows:

[0098] Example

[0099] A method for road extraction from remote sensing images based on dual encoders and frequency sensing includes the following steps:

[0100] Step S1: Data Preparation and Preprocessing

[0101] High-resolution remote sensing imagery (such as GF-2 satellite imagery) covering different regions and scenes was acquired. To ensure consistency of model input, the raw imagery underwent standard preprocessing, including radiometric calibration, geometric correction, and atmospheric correction. Subsequently, the imagery was cropped or scaled to the network's standard input size, such as 512×512 pixels. If the input was a single-channel image, it was copied to a three-channel image at the start of the network's forward propagation to fit the input requirements of the SAM image encoder. Data was organized using a custom ImageFolder dataset class, and PyTorch's DataLoader was used for batch loading and random shuffling, providing a high-quality data stream for model training.

[0102] Step S2: Model Initialization and Dual Encoder Construction

[0103] The model construction begins with the initialization of the dual encoder structure, the core of which is shown in the appendix. Figure 2 As shown. The specific implementation is as follows:

[0104] Main semantic encoder: A ResNet-34 pre-trained on ImageNet is used as the backbone network to extract task-related multi-scale local geometric features. The encoder outputs feature maps at four levels with spatial resolutions of 1 / 4 (e1), 1 / 8 (e2), 1 / 16 (e3), and 1 / 32 (e4) of the input, respectively.

[0105] Image encoder description: An image encoder based on a SAM model (lightweight MobileSAM is used in this embodiment) is introduced as a second branch providing a robust global prior. To ensure its strong generalization ability is not compromised, all weight parameters of this encoder are frozen by setting p during training. The images input to this branch need to be pre-sampled to a fixed size of 1024×1024 using bilinear interpolation.

[0106] Feature fusion: The features output by the SAM encoder (e.g., 256-dimensional) first pass through a projection module (consisting of 1×1 convolution, batch normalization, and ReLU activation function) to transform their channel count to match the high-level features of the main encoder. (512-dimensional) Consistent. Subsequently, the projected SAM features are compared with those obtained by element-wise addition. By fusing, enhanced fusion features are obtained. This design cleverly combines the general semantic priors of SAM with the specific geometric features of the task backbone.

[0107] Step S3, Context Enhancement Module

[0108] To expand the receptive field without sacrificing spatial resolution, thereby better perceiving the global contextual information of the road, feature fusion is employed. Then, a multi-scale dilated convolutional module (Dblock) is connected, the structure of which is shown in the attached figure. Figure 3 As shown, this module contains multiple parallel 3×3 dilated convolutional layers with different dilation rates (e.g., 1, 2, 4, 8), which can effectively aggregate multi-scale contextual information and enhance the model's robustness to complex scenes and background disturbances. The outputs of each branch are aggregated through residual connections.

[0109] Step S4: Multi-stage noise perception self-training

[0110] The decoding path is crucial for recovering the high-resolution road mask. This embodiment introduces an innovative frequency-aware fusion mechanism, as shown in the attached figure. Figure 3 As shown.

[0111] Frequency-Aware Fusion Module: This module operates sequentially at three decoding scales: 1 / 16, 1 / 8, and 1 / 4 (corresponding to modules ff4, ff3, and ff2). It takes high-resolution features (rich in high-frequency edge details) from the encoding path and low-resolution, upsampled, strong semantic features (rich in low-frequency regional information) from deeper layers as dual inputs. Its core lies in dynamically weighting and fusing high-frequency details and low-frequency semantics through a content-adaptive mechanism, aiming to suppress intra-class inconsistencies within road regions while sharpening boundaries blurred by downsampling. During forward propagation, bilinear interpolation ensures strict spatial alignment of the input feature maps, avoiding fusion misalignment.

[0112] Topology-aware decoding block: as attached Figure 4 As shown, the features processed by the frequency-aware fusion module are fed into the decoding block. This block first uses 1×1 convolutions for channel dimensionality reduction to reduce computation. Subsequently, the features are passed in parallel through three star-shaped convolutional structures of different scales (kernel sizes of 5, 9, and 13). These structures explicitly model the linear topological characteristics of the road through strip convolutions in the horizontal, vertical, and diagonal directions, effectively mitigating the breakage problem caused by occlusion. After the multi-scale features are concatenated and fused by 1×1 convolutions, the residuals are added to the features reduced by the main branch, and then a 2x upsampling is performed through transposed convolutions to gradually restore the spatial resolution. This process starts with the generation of d4, and d3, d2, etc. are generated sequentially, finally obtaining the high-resolution feature d1.

[0113] Step S5: Predicted Output and Post-processing

[0114] Finally, the high-resolution feature d1 output by the decoder passes through an output header, which typically consists of transposed convolutional and standard convolutional layers, mapping the number of channels to the number of classes (1 in this example). A sigmoid activation function is then used to obtain a probability map (range 0-1) indicating whether each pixel belongs to a road. During the inference phase, a threshold (e.g., 0.5) is set to adjust the probability. Figure 2The values ​​are then converted into the final road mask. To further optimize the results, selective post-processing operations can be performed, such as morphological opening to remove minor noise, or skeletonization and broken connection algorithms can be used to improve the topological connectivity of the road network.

[0115] Step S6: Predicted Output and Post-processing

[0116] The model is trained end-to-end, with the following key configurations:

[0117] Loss function: A combination of binary cross-entropy loss and Dice loss is used (Loss = L_BCE + (1 - L_Dice)). This combination can simultaneously take into account pixel-level classification accuracy and the overall overlap of road regions, effectively addressing the class imbalance problem between roads and the background.

[0118] Optimizer and Learning Rate: The Adam optimizer is used, with an initial learning rate set to 2e-4. A step decay strategy is employed: if the validation set performance does not improve over several consecutive training epochs, the learning rate is reduced by a factor (e.g., 5 times) to promote convergence.

[0119] Data augmentation: Various data augmentation techniques were applied during training to improve the model's generalization ability, including random rotation (such as 30°, 60°, 90°, 120°, 150°), horizontal / vertical flipping, color jitter, brightness and contrast adjustment, etc.

[0120] Training monitoring and early stopping: Monitor the loss value during training. If the performance does not improve within several consecutive periods (e.g., 6 periods), the early stopping mechanism is triggered, and the model weights with the best performance on the validation set are saved.

[0121] Step S7, Enhancement and Inference During Testing

[0122] During the model inference phase, to improve the stability and accuracy of the results, a test-time enhancement strategy can be adopted. Specifically, the input image is subjected to various transformations (such as rotation by 90°, 180°, 270°, horizontal flip, and vertical flip), and the transformed images are input into the network separately to obtain prediction results. Then, the results are subjected to corresponding inverse transformations and averaged or fused by voting. This strategy helps the model "observe" the image from different perspectives, reducing misjudgments caused by a single perspective, and especially improving the extraction effect of direction-sensitive road structures.

[0123] The construction and verification process of the road extraction method based on dual encoders and frequency sensing in this embodiment is described as follows:

[0124] 1. Selection of study area and construction of dataset

[0125] To verify the generalization and robustness of the proposed method, three representative road extraction datasets were selected for evaluation: the HF road dataset, constructed based on domestic Gaofen-2 (GF-2) satellite imagery, covering urban and suburban areas of Hefei City, containing 12,628 512×512 pixel samples with a spatial resolution of 1 meter. This dataset is characterized by complex urban structures and numerous building shadows, overpasses, and vegetation occlusion. The DeepGlobe dataset, a public road extraction dataset, covers rural and suburban areas in Thailand, India, and Indonesia, containing 6,226 1024×1024 pixel images, processed into 24,904 512×512 samples with a spatial resolution of 0.5 meters. This dataset presents vegetation occlusion and rural roads as major challenges. The Massachusetts dataset, a public road dataset, covers various terrains in Massachusetts, USA, containing 1,171 1500×1500 pixel aerial images, processed into 11,710 512×512 samples with a spatial resolution of 1 meter. This dataset includes urban, suburban, rural, and mountain roads, and exhibits seasonal shading variations.

[0126] All datasets were randomly divided into training, validation, and test sets in a ratio of approximately 70%:27%:3% to ensure the fairness and reproducibility of the experiments.

[0127] 2. Implementation of Dual Encoder and Frequency Sensing Fusion Network

[0128] The specific implementation of the core network FADENet in this embodiment is as follows:

[0129] Dual encoder fusion: such as Figure 2 As shown, the SAM image encoder provides robust global prior features, which are deeply fused with multi-scale local semantic features (e1-e4) extracted by the ResNet-34 backbone. Before feature fusion, projection and precise spatial size alignment are used to ensure effective information complementarity.

[0130] Frequency sensing fusion module: This module ( Figure 3 By adaptively generating convolution kernels to simulate frequency domain filtering, high-frequency (road edge) and low-frequency (road area) information are processed differently, effectively solving the problems of intra-class inconsistency and boundary displacement in traditional jump-connection fusion.

[0131] Multi-directional topology-aware aggregation: in the decoding block ( Figure 4 Multi-scale star convolution explicitly captures the linear structure of roads through strip convolution in four directions (horizontal, vertical, and two diagonals), greatly enhancing the connectivity of roads in occluded scenes.

[0132] 3. Multi-stage optimization training strategy

[0133] The model training employs a rigorous multi-stage strategy to ensure stability and performance:

[0134] Loss function and optimization: We use a BCE+Dice composite loss, Adam optimizer, initial learning rate of 2e-4, and learning rate decay.

[0135] Data augmentation: A variety of data augmentation techniques were implemented, including random rotation (multiple angles), horizontal / vertical flipping, and color jitter, to improve the model's adaptability to different lighting and angle conditions.

[0136] Parameter settings: SAM encoder weights are frozen, and only the main encoder, Dblock, decoder, and output head are trained to ensure effective task adaptive learning while incorporating prior knowledge.

[0137] 4. Accuracy Assessment and Comparative Analysis

[0138] To comprehensively verify the overall performance of this embodiment, a system evaluation was conducted from three aspects: quantitative accuracy, qualitative quality, and the effectiveness of core components. Experimental results show that this embodiment achieves road extraction performance comparable to or even better than fully supervised methods without any manual annotation.

[0139] In terms of quantitative accuracy, this method achieved excellent performance on the test sets of all three datasets. Specifically, on the HF dataset, this embodiment achieved an Intersection over Union (IoU) of 73.64% and an F1 score of 84.82%; on the DeepGlobe dataset, the IoU reached 72.98% and the F1 score reached 84.39%; and on the Massachusetts dataset, the IoU was 63.63% and the F1 score was 77.77%. This embodiment demonstrated significant advantages across all key metrics, proving its powerful extraction capability and generalization ability.

[0140] In terms of qualitative quality, the visualization results further confirm the effectiveness of this method. For example... Figure 5 As shown, this method effectively resists building shadows and overpass occlusion in complex urban scenes, maintaining the topological continuity of roads; it can clearly distinguish the road-vegetation boundaries in densely vegetated rural scenes; and it can accurately extract roads and suppress false positives in scenes with spectral confusion and shadow variations. The extracted road network has good connectivity and accurate boundaries.

[0141] Based on the above evaluation results, this embodiment not only meets the practical requirements in terms of overall accuracy, but its core algorithm components (dual encoder, frequency-aware fusion, and topology-aware aggregation) and training strategies have also been verified to have significant technical advancements and performance contributions, providing an efficient, robust, and universal solution for road extraction from high-resolution remote sensing images.

[0142] The preferred embodiments of the present invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to specific implementations. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention. The invention is limited only by the claims and their full scope and equivalents.

Claims

1. A method for road extraction from remote sensing images based on dual encoders and frequency sensing, characterized in that, Includes the following steps: S1. Construct a road extraction dataset, including data acquisition, preprocessing, enhancement, and segmentation; S2. Design a dual encoder architecture that integrates the SAM image encoder and the D-LinkNet backbone network; S3. Design a multi-directional topology sensing aggregation module to enhance road connectivity through multi-scale four-directional strip convolution; S4. Design an adaptive frequency sensing fusion module to optimize boundary accuracy through content-adaptive high-pass and low-pass filtering; S5. Construct a multi-scale enhanced decoder, integrating the modules from steps S3 and S4 for feature decoding; S6. Model training and optimization, using a hybrid loss function and learning rate decay strategy; S7. Model performance evaluation, including quantitative indicator analysis and qualitative visualization verification.

2. The method for road extraction from remote sensing images based on dual encoders and frequency sensing according to claim 1, characterized in that, High-resolution remote sensing images were acquired, cropped into 512×512 pixel RGB images, and road pixels were manually labeled; data augmentation was performed using random rotation, horizontal flipping, and vertical flipping; the dataset was divided into training, validation, and test sets according to the proportions, and corresponding file lists and metadata files were generated.

3. The method for road extraction from remote sensing images based on dual encoders and frequency sensing according to claim 1, characterized in that, The dual-encoder architecture comprises a SAM image encoder and a D-LinkNet encoder operating in parallel. The SAM image encoder, based on a pre-trained ViT architecture, extracts global semantic features and freezes parameters to maintain generalization ability. The D-LinkNet encoder, based on ResNet-34, uses dilated convolutions to expand the receptive field and extract local geometric features. The dual-encoder features are aligned in spatial dimensions through projection and downsampling, and fused through element-wise addition and 1×1 convolutions, as expressed in the formula: ； in, Features of D-LinkNet Features of SAM This is a projection operation.

4. The method for road extraction from remote sensing images based on dual encoders and frequency sensing according to claim 1, characterized in that, The multi-directional topology-aware aggregation module captures the linear topology of roads through multi-scale strip convolution, using four-directional dilated strip convolution at three scales. It transforms diagonal convolutions into horizontal convolutions through coordinate transformation, achieving efficient multi-directional feature extraction. Multi-scale features are fused through channel stitching and 1×1 convolution to enhance road connectivity under occlusion, as expressed by the formula: 。 5. The method for road extraction from remote sensing images based on dual encoders and frequency sensing according to claim 1, characterized in that, The adaptive frequency sensing fusion module separates high- and low-frequency information through content adaptive filtering; The adaptive low-pass generator generates a low-pass filter kernel through the content encoder, smoothing the internal features of the road and reducing intra-class inconsistencies; the adaptive high-pass generator generates a high-pass filter kernel, enhancing boundary details and suppressing background noise; the fusion process is expressed by the following formula: ； in, This is a Qualcomm characteristic. It is a low-pass characteristic. These are the original high-frequency features.

6. The method for road extraction from remote sensing images based on dual encoders and frequency sensing according to claim 1, characterized in that, The multi-scale enhanced decoder progressively upsamples and fuses encoder features: each stage of the decoder integrates AFAF and MTAA modules to recover spatial details through skip connections; the AFAF module fuses high and low resolution features in each layer of the decoder to optimize boundary localization; the MTAA module enhances the continuity of road topology and reduces breaks caused by occlusion; the final output is upsampled to the original resolution through transposed convolution to generate a road probability map.

7. The method for road extraction from remote sensing images based on dual encoders and frequency sensing according to claim 1, characterized in that, Model training uses a hybrid loss function: ； For binary cross-entropy loss, The loss is Dice; the optimizer uses Adam with an initial learning rate of 2×10⁻⁶. -4 A step size decay strategy is adopted; the batch size is 8, and the training cycle is stopped early based on the performance of the validation set.

8. The method for road extraction from remote sensing images based on dual encoders and frequency sensing according to claim 1, characterized in that, Model evaluation metrics include intersection-over-union ratio, F1 score, precision, recall, and overall accuracy. Comparative experiments and ablation studies are conducted on the dataset constructed in step S1 to verify the model's advantages in road connectivity and boundary accuracy in occluded scenarios.

Images