Semi-supervised optical remote sensing image road extraction method based on feature calibration

Abstract

The invention belongs to the technical field of remote sensing image analysis, and discloses a semi-supervised optical remote sensing image road extraction method based on feature calibration, and the method comprises the steps: generating a road extraction result of a to-be-processed optical remote sensing image through a road extraction model; the road extraction model is obtained by the following steps of: constructing a student sub-network and a teacher sub-network of an encoder-decoder network architecture based on a Mean-Tester semi-supervised framework; synchronously using labeled and unlabeled data to train the student sub-network and the teacher sub-network; after the training is completed, taking the teacher sub-network as a road extraction model; wherein a DIMT module is arranged on a bottleneck layer of an encoder-decoder network architecture; according to the method, bidirectional interaction and global complementation of labeled data and unlabeled data on a feature level are realized, the problem of feature distribution mismatch in semi-supervised learning is solved, the quality of pseudo labels is improved, and high-precision semi-supervised road extraction is realized.

Description

Technical Field

[0001] This invention belongs to the field of remote sensing image analysis technology, specifically relating to a semi-supervised optical remote sensing image road extraction method based on feature calibration. Background Technology

[0002] Road extraction from optical remote sensing images is a core technology for acquiring geospatial road network information and has important applications in fields such as autonomous driving, traffic monitoring, and smart cities. Traditional methods, based on morphological and texture analysis, are inefficient when processing large-area, high-resolution images. With the development of deep learning, fully supervised semantic segmentation models (such as DLinkNet and TransRoadNet) have significantly improved accuracy, but their training is extremely dependent on a large number of pixel-level labeled samples, resulting in high labeling costs and a tendency to overfit due to labeling noise.

[0003] To reduce annotation costs, semi-supervised road extraction methods (such as ADVENT and SemiRoadExNet) have been proposed. These methods use a small amount of labeled data and a large amount of unlabeled data for joint training. However, during the training process, the pseudo-labels generated by the unlabeled data often contain a lot of noise and errors. These low-quality pseudo-labels, as supervision signals, can mislead the training of student models, thereby reducing the accuracy of road extraction. Summary of the Invention

[0004] The purpose of this invention is to provide a semi-supervised road extraction method based on feature calibration for optical remote sensing images, so as to improve the quality of pseudo-labels and thus achieve high-precision semi-supervised road extraction.

[0005] The present invention adopts the following technical solution: A semi-supervised optical remote sensing image road extraction method based on feature calibration includes: generating road extraction results of the optical remote sensing image to be processed using a road extraction model; The road extraction model is obtained in the following way: Based on the Mean-Teacher semi-supervised framework, student subnetworks and teacher subnetworks with the same encoder-decoder network architecture are constructed. The student subnetwork and the teacher subnetwork are trained simultaneously using labeled and unlabeled data; the labeled data includes optical remote sensing images and their corresponding road segmentation binary maps, and the unlabeled data includes optical remote sensing images. After training, the teacher subnetwork is used as the road extraction model; In the bottleneck layer of the encoder-decoder network architecture, a DIMT module is set up to realize feature interaction and global feature complementarity between labeled and unlabeled data in the student subnetwork and the teacher subnetwork.

[0006] The beneficial effects of this invention are as follows: This invention constructs a road extraction model based on the Mean-Teacher semi-supervised framework, and embeds the DIMT module in the bottleneck layer of the Mean-Teacher semi-supervised framework, realizing bidirectional interaction and global complementarity between labeled and unlabeled data at the feature level, solving the problem of feature distribution mismatch in semi-supervised learning, improving the quality of pseudo-labels, and thus achieving high-precision semi-supervised road extraction. Attached Figure Description

[0007] Figure 1 This is a diagram illustrating the overall architecture of the road extraction model in this invention. Figure 2 This is a comparison diagram between the road extraction model of the present invention and the road extraction model in the prior art; Figure 3 This is a schematic diagram of the DIMT module in this invention; Figure 4 This is a schematic diagram of the RSM module in this invention; Figure 5 This is a schematic diagram of the RPC module in this invention; Figure 6 Visualization results of comparative experiments on the Massachusetts road dataset; Figure 7 Visualization results of comparative experiments on the DeepGlobe road dataset; Figure 8 Visualization results of comparative experiments on the CHN6-CUG road dataset. Detailed Implementation

[0008] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0009] Existing semi-supervised road extraction methods often suffer from the following problems: (1) Feature distribution mismatch: In existing semi-supervised methods, labeled data and unlabeled data are trained separately, which does not make full use of the global feature interaction between the two, resulting in low quality of pseudo-labels for unlabeled data, which can easily mislead model training; (2) Class imbalance problem: The proportion of roads (foreground) in remote sensing images is much lower than that of the background. Existing methods have difficulty learning the features of small roads (such as dead ends and narrow roads), which easily leads to the omission of small roads; (3) Occlusion and edge bias: Occlusion by buildings and trees and high similarity between roads and background can cause road edge prediction to be blurred. Existing methods (such as Transformer-based methods) can capture global information, but they have secondary computational complexity. Methods using Mamba-type models are not optimized for the slender and multi-directional distribution characteristics of roads. (4) Weak generalization ability of the model: Under extremely low labeling rates (such as 5%), existing methods are unable to mine the potential information of unlabeled data, resulting in poor road extraction integrity in complex scenarios (such as sandy areas and densely built-up areas).

[0010] Therefore, there is an urgent need for a semi-supervised road extraction technique that can achieve feature calibration, alleviate class imbalance, optimize edge prediction, and be computationally efficient under low annotation rates, so as to reduce annotation costs and improve road extraction performance in complex scenarios.

[0011] like Figure 1 and Figure 2 As shown, this invention proposes a semi-supervised road extraction method based on feature calibration in optical remote sensing images. A road extraction model is constructed based on a Mean-Teacher semi-supervised framework. First, a dual-interaction Mamba-Transformer module (DIMT module) is integrated into the road extraction model. By learning global interactions, it expands the scene perception range, effectively improving the feature distribution mismatch between unlabeled and labeled data, achieving global feature alignment, and thus obtaining more complete road topology and main road information. Second, a road prototype calibration module (RPC module) is adopted, using road prototype features from labeled data to guide unlabeled data, improving road extraction performance in class-imbalanced scenarios, avoiding the omission of small target roads, and achieving road feature alignment. Finally, a road SMamba module (RSM module) is introduced, which learns global and local features simultaneously through gating fusion, thereby alleviating the problem of blurred road edge prediction caused by occlusion and high similarity under extremely low label rates. Even using only 5%-20% of labeled samples, high-precision and high-completeness road extraction can still be achieved.

[0012] The method of the present invention is as follows: A semi-supervised method for road extraction from optical remote sensing images based on feature calibration includes: generating road extraction results from the optical remote sensing images to be processed using a road extraction model.

[0013] The road extraction model is obtained in the following way: Based on the Mean-Teacher semi-supervised framework, student subnetworks and teacher subnetworks with the same encoder-decoder network architecture are constructed. The student subnetwork and the teacher subnetwork were trained simultaneously using labeled and unlabeled data; the labeled data included optical remote sensing images and their corresponding road segmentation binary maps, and the unlabeled data included optical remote sensing images. After training, the teacher subnetwork was used as the road extraction model.

[0014] In the bottleneck layer of the encoder-decoder network architecture, a DIMT module is set up to realize feature interaction and global feature complementarity between labeled and unlabeled data in the student subnetwork and the teacher subnetwork.

[0015] Specifically, in the Mean-Teacher semi-supervised framework, the student sub-network updates parameters in real time through gradient backpropagation and is responsible for learning the supervised loss and consistency loss of the labeled data; the teacher sub-network updates parameters through the exponential moving average algorithm and generates pseudo-labels for unlabeled data (probability ≥0.5 for roads, <0.5 for background), ensuring the stability of the pseudo-labels.

[0016] The update formula for the exponential moving average algorithm is: , in, These are the weight parameters updated after each gradient descent iteration of the student subnetwork. The weight parameters of the teacher subnetwork are smoothed using the exponential moving average algorithm; This represents the weighting factor, verified through numerous experiments. Setting it to 0.9 controls the weighting of historical parameters. The weighting coefficients for student parameters are used to determine the degree to which the current student parameters contribute to the updating of teacher parameters.

[0017] like Figure 3 As shown, the DIMT module includes an interactive Mamba unit and an interactive Transformer unit.

[0018] Interactive Mamba units are used to flatten the features output by the encoder into a one-dimensional sequence along multiple spatial directions and perform feature interactions on the one-dimensional sequence.

[0019] The interactive Transformer unit is used to perform cross-attention calculation on the features output by the interactive Mamba unit to achieve global feature complementarity.

[0020] The DIMT module is configured during the training phase as follows: Receive labeled and unlabeled data features output from the encoder; The labeled and unlabeled data features are sequentially passed through interactive Mamba units and interactive Transformer units to achieve feature interaction and global feature complementarity between labeled and unlabeled data.

[0021] Configured during the inference phase as follows: Receive the optical remote sensing image features to be processed from the encoder output; The optical remote sensing image features to be processed are sequentially passed through interactive Mamba units and interactive Transformer units to enhance the features of the optical remote sensing image features to be processed.

[0022] In this invention, the DIMT module is used during the training phase to address the global feature distribution mismatch between labeled and unlabeled data. Specifically: Feature concatenation and splitting: Concatenate labeled data and unlabeled data along dimension 0, then input them into the encoder to obtain intermediate features. Split the intermediate features along dimension 0 into labeled data features and unlabeled data features.

[0023] Interactive Mamba processing: The labeled and unlabeled data features are flattened into 8-directional one-dimensional sequences (positive and negative directions of the main diagonal, positive and negative directions of the anti-diagonal, positive and negative directions of the horizontal direction, and positive and negative directions of the vertical direction); the one-dimensional sequences of labeled and unlabeled data features are input into the interactive selective scanning state space sequence model (IS6 model) for selective scanning to achieve feature interaction; the 8-directional scanning results of the one-dimensional sequences of labeled and unlabeled data features are reconstructed into two-dimensional feature maps, and summed respectively.

[0024] The IS6 model employs a State-Space Model (SSM) to achieve global feature learning with linear complexity, avoiding the secondary computational overhead of the Transformer. Furthermore, it introduces a shortcut mechanism to enable cross-data interaction, configured during the training phase as follows: When performing selective state scanning on a one-dimensional sequence of labeled data features, embed a one-dimensional sequence of unlabeled data features; When performing selective state scanning on a one-dimensional sequence of unlabeled data features, a one-dimensional sequence of labeled data features is embedded to achieve bidirectional feature interaction.

[0025] The scanning formula for the IS6 model is: , , in, The training state matrix contains the system's internal hidden states, used to memorize historical information and represent long-range dependencies; The feature vectors input to the model; This is the state transition matrix, used to control how the state changes. change; For input mapping terms, used to... The bias or mapping injected into the state space; The output feature vector of the model; The projection matrix from the state to the output is used to project the hidden state. Mapping to the output space determines which state dimensions contribute to the output; The output modulation matrix is ​​used to control the importance of different state channels; The input direct connection is similar to a residual connection, used to preserve local information and improve training stability.

[0026] , , , Linear mapping is achieved through 3×3 convolutional layers.

[0027] Interactive Transformer processing: The summation results of labeled and unlabeled data features are linearly mapped to the query through 1×1 convolutions. ),key( ),value( By using cross-attention calculation, global feature complementarity between labeled and unlabeled data is achieved, thereby improving the quality of pseudo-labels.

[0028] Attention is calculated using the following formula: , in, The query vector, obtained by encoding the input sequence, represents the feature vector of each position "actively matching other positions," used in... Search for relevant information in the middle; The dimension of the query vector; The key vector is a key matrix obtained by encoding the input sequence, representing the feature vector "matched" at each position, and... Pairing similarity is calculated; It is a value vector, a value matrix obtained by encoding the input sequence, which stores the original information of each position and is the object of final attention weighting; This is the bias matrix.

[0029] The output of the interactive Transformer unit is used as the input to the decoder for further processing.

[0030] In this invention, by embedding a DIMT module in the bottleneck layer, it enables real-time, bidirectional interactive scanning and attention complementarity of features of labeled and unlabeled data during the training phase. This design directly affects the feature encoding process, aligning the feature distributions of the two types of data from the source, thereby significantly improving the accuracy and reliability of pseudo-labels generated by the teacher model. This provides a key technical means to solve the performance bottleneck caused by data distribution mismatch in semi-supervised learning.

[0031] like Figure 4As shown, in the skip connections of the encoder-decoder network architecture, an RSM module for fusing global and local features is embedded. The RSM module includes a global branch, a local branch, and a gated fusion unit.

[0032] The global branch uses a selective scan state space sequence model (S6 model) to extract global features.

[0033] Local branches are extracted using multi-directional strip convolution to extract local features.

[0034] Gated fusion units are used to generate adaptive weights to fuse global and local features.

[0035] The RSM module is configured during the training phase as follows: Receive skip connection features from the encoder, which include skip connection features of labeled data and skip connection features of unlabeled data; Global features of skip connection features are extracted through global branches; Local features of skip connection features are extracted through local branches; The global and local features of the skip connection feature are adaptively fused by the gated fusion unit, and the optimized feature is output.

[0036] Configured during the inference phase as follows: Receive skip connection features of the optical remote sensing image to be processed from the encoder; Global features of the skip connection features of the optical remote sensing image to be processed are extracted by global branching. Local features of skip connectivity features in the optical remote sensing image to be processed are extracted by local branching. The global and local features of the skip connection features of the optical remote sensing image to be processed are adaptively fused by the gated fusion unit, and the optimized features are output.

[0037] In this invention, the RSM module is used to optimize occlusion and edge prediction during the training phase, specifically: Global Branch: The skip connection features from the encoder are sequentially passed through 1×1 convolutions (linear mapping) and depthwise separable convolutions (DWConv), then input into the S6 model. After scanning in 8 directions, the features are reconstructed and summed, and fused with the input skip connection features to obtain global features, thus solving the problem of feature capture in occluded regions. The internal structure of the S6 model is consistent with the basic part of the IS6 model. It uses a state-space model (SSM) to achieve global feature learning with linear complexity, avoiding the secondary computational overhead of the Transformer. However, the S6 model does not include a shortcut connection mechanism for cross-data interaction.

[0038] Local branch: The skip connection features from the encoder are processed in parallel through 4-directional strip convolution (horizontal, vertical, top left to bottom right, bottom left to top right) to extract local features; where the convolution kernels of the 4-directional strip convolution are: 9×1 or 1×9 in the horizontal / vertical direction, and 9×1 strip kernel in the top left to bottom right / bottom left to top right diagonal direction, with a total number of directions m=4.

[0039] The convolution formula is: , in, The feature map output by the local branch is a feature map that combines the results of strip convolution in four directions; It is a weighted sum of the convolution results in four directions. For direction index ( =1, 2, 3, 4 correspond to the four directions: horizontal, vertical, from top left to bottom right, and from bottom left to top right, respectively. For the first Strip convolution operations in each direction use the corresponding convolution kernel (e.g., 9×1 / 1×9 / diagonal 9×1). The feature map is the input for the local branch; For the first The directional weights / direction masks are used to weight the convolution results in that direction, controlling the contribution of features in different directions.

[0040] Gated fusion unit: generates gating coefficients using a trainable matrix and activation function. , ,according to Integrating global and local features; among which, This is the output feature map of the gated fusion unit. , The gating weights are used for global features, and the output is a probability value in the interval (0, 1), which is used to control the proportion of global features in the fusion; the sigmoid is an activation function that maps the input to the interval (0, 1), and the output can be used as the probability value of the weights. These are learnable 3×3 convolution kernel parameters used for spatial feature extraction and transformation of input features; It is a learnable bias term used to linearly shift the convolutional features, thereby enhancing the model's expressive power. The feature map output for the global branch; ,in The gating weights for local features, and Complementarity ensures that the weights of global and local features sum to 1, achieving dynamic balance fusion.

[0041] The gated fusion unit is used to adaptively adjust the weights of global and local features to optimize road edge prediction.

[0042] In this invention, by embedding an RSM module in the skip connection, its global branch and local branch capture the global features and local details of the road, respectively, and then adaptively fuse them through a gating unit. This multi-scale feature fusion mechanism designed for road geometry can handle scenes with building and tree occlusion and high road-background similarity, improving the road edge prediction accuracy by 10%-15%. It effectively enhances the robustness of the model to complex contexts (such as occlusion and shadows) and reduces breakage and blurring phenomena.

[0043] like Figure 5 As shown, an RPC module for feature calibration based on prototype features is set between the student subnetwork and the teacher subnetwork; the RPC module includes a road prototype feature library (RPFB unit) and a similarity calculation unit.

[0044] The RPFB unit is used to store road prototype features from the labeled data extracted from the teacher subnetwork.

[0045] The similarity calculation unit is used to calculate the similarity between the unlabeled data features in the student subnetwork and the road prototype features, and to calibrate the unlabeled data features in the student subnetwork based on the similarity.

[0046] In this invention, the RPC module is used to alleviate class imbalance in order to preserve smaller roads, specifically: RPFB unit: Extracts road feature points (label value = 1) from the teacher subnetwork auxiliary output layer (feature dimension 32×512×512), generates prototype features through global average pooling, and maintains length using a first-in-first-out (FIFO) mechanism. The feature library provides feature templates for small roads.

[0047] The formula for calculating prototype features is: , in, This represents the road pixels in the labeled data. This represents the total number of pixels on the road. This represents the image index, that is, the input image number to which the current pixel belongs. Represents the pixel index, that is, the pixel index at the 1st position. The road pixel number within the image.

[0048] Similarity calculation unit: Obtain unlabeled data features from the student subnetwork auxiliary output layer, calculate the cosine similarity between each spatial location of the feature and the prototype feature in the RPFB unit, obtain a similarity map after Softmax normalization, aggregate along the channel dimension to obtain the calibration result, and guide the unlabeled data to learn road features through pseudo-label constraints of the teacher subnetwork to avoid missing small roads.

[0049] In this invention, an RPC module is introduced between the student and teacher networks to dynamically build and maintain a road prototype feature library extracted from labeled data. This module is calibrated by calculating the similarity between the features of unlabeled data and the prototype library, providing the model with clear "road" feature references. This guides the model to better identify road segments with blurred appearance, small shape, or rare features, effectively extracting small roads such as dead ends and narrow roads, and solving the problem of missing small roads caused by class imbalance.

[0050] The training process employs a combined loss function, which includes a consistency loss to constrain the difference between the prediction results and the supervision signal, an entropy minimization loss to improve the model's confidence in predicting unlabeled data, and a prototype loss to constrain the consistency between unlabeled data features and road prototype features.

[0051] Specifically, consistency loss ( ) is a weighted combination of Focal loss (which resolves class imbalance) and Dice loss (which focuses on foreground paths), used to constrain pixel-level classification accuracy.

[0052] The formula for Focal loss is: , in, Focal loss function is used to address class imbalance and hard sample mining problems; This is a category balance coefficient used to control the loss weights of different categories in this invention. ; This represents the model's predicted probability of the true class. Difficult sample adjustment factor, To focus on parameters used to reduce the loss weight of easily separable samples and amplify the loss contribution of difficult samples, this invention... ; This is the standard cross-entropy loss term, used to measure the difference between the predicted probability and the true label.

[0053] The formula for Dice loss is: , in, The Dice loss function; To predict probabilities; For tag values; This is a smoothing term used to avoid calculation errors caused by a denominator of 0, while also ensuring numerical stability. .

[0054] Entropy minimization loss ( This is used to reduce the prediction entropy of unlabeled data and improve the confidence of pseudo-labels.

[0055] Prototype loss ( The calculation measures the consistency between the features extracted by the student subnetwork from the unlabeled data and the road prototype features stored in the RPC module from the teacher subnetwork. This consistency is used to constrain the calibration of unlabeled data and prototype features, ensuring road feature consistency. The formula is: , in, The prototype loss is used to ensure that the features extracted by the student subnetwork are consistent with the road prototype features generated by the teacher subnetwork, thereby achieving feature calibration under semi-supervised learning. The aggregated features extracted by the student subnetwork from unlabeled data are the road feature representations learned by the student network from the input unlabeled images. The pseudo-label prototype features are derived from the road prototype features generated by the teacher subnetwork stored in the RPC module, and serve as a supervisory signal to guide the feature learning of the student network. is the square of the L2 norm (mean square error), used to measure the Euclidean distance between two feature vectors. The smaller the value, the more consistent the features are.

[0056] The total loss is: ,in, The weighting coefficient for entropy loss is used to control the proportion of entropy loss in the total loss. In this invention... ; The prototype loss is a weighting coefficient used to control the proportion of prototype loss in the total loss. In this invention... .

[0057] The road extraction method of the present invention is as follows: Load the trained road extraction model; Input the optical remote sensing image to be processed (resolution ≥ 512×512, 3 channels). If the size of the optical remote sensing image to be processed is greater than 512×512, use a sliding window to crop it into a 512×512 sub-image. The subgraph is input into the model, which processes it sequentially through the DIMT module and the RSM module, and outputs a road prediction probability map of the subgraph. The road prediction probability map is segmented by threshold (roads are defined as having a probability ≥ 0.5), and the sub-maps are stitched together (to eliminate duplicate predictions in overlapping window areas). The final output is a binary map of the extracted roads (white for roads, black for the background), which can be further used for road vector map generation or traffic monitoring systems.

[0058] The effects of the present invention will be explained in detail below with reference to simulation experiments.

[0059] 1. Experimental environment configuration: Hardware environment: 2 NVIDIA GeForce RTX 3050 GPUs (12GB VRAM), Intel Xeon Gold 6230 CPU, 64GB RAM; Software environment: Linux operating system, Python 3.10, PyTorch 2.0 framework, CUDA 11.7; Optimizer and Training Strategy: Adam optimizer, initial learning rate 0.001, multinomial decay strategy. epoch represents the number of training rounds, total epoch represents the total number of training rounds, total epoch = 150, final learning rate 0.000005); early stopping strategy (training is terminated if performance declines after 5 consecutive epochs); batch size is 4.

[0060] 2. Datasets and Data Augmentation: The effectiveness of this invention was verified using three publicly available optical remote sensing road datasets: The DeepGlobe dataset consists of 6226 images of 1024×1024×3 pixels (0.5m / pixel), divided into a training set of 4980 images and a test set of 1246 images. Massachusetts dataset: 1171 images of 1500×1500×3 pixels (1.5m / pixel), including 1108 training images, 14 validation images, and 49 test images; CHN6-CUG dataset: 4511 images of 512×512×3, including 3608 training images and 903 test images, covering various road types in urban and rural areas.

[0061] Data augmentation: The images from the DeepGlobe and Massachusetts datasets were cropped to 512×512 pixels, and random cropping, horizontal / vertical flipping (geometric augmentation), and brightness / contrast / saturation adjustment (morphological augmentation) were applied to avoid model overfitting.

[0062] 3. Analysis of experimental results: The method of this invention is combined with widely used fully supervised models in the field of road extraction, such as DLinkNet (L. Zhou, C. Zhang, and M. Wu, “D-linknet: Linknet with pretrained encoder and dilated convolution for high resolution satellite imagery road extraction,” in IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recogn. Workshops, 2018, pp. 192–1924) and TransRoadNet (Z. Yang, D. Zhou, Y. Yang, J. Zhang, and Z. Chen, “Transroadnet: A novel road extraction method for remote sensing images via combining high-level semantic feature and context,” IEEE Geosci. Remote Sens. Lett., vol. 19, pp. 1–5, 2022.), as well as semi-supervised models, such as AllSpark (H. Wang, Q. Zhang, Y. Li, and X. Li, “Allspark: Reborn labeled features from unlabeled intransformer for semi-supervised semantic features). segmentation," in Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition, 2024, pp.3627–3636.), SemiRoadExNet (H. Chen, Z. Li, J. Wu, W. Xiong, and C. Du, "Semiroadexnet: A semi- supervised network for road extraction from remotesensing imagery via adversarial learning," ISPRS-J. Photogramm. Remote Sens., vol.(198, pp. 169–183, 2023.), CPS (X. Chen, Y. Yuan, G. Zeng, and J. Wang, "Semi-supervised semantic segmentation with cross pseudo supervision," in Proc IEEE Comput Soc Conf Comput Vision Pattern Recognit, 2021, pp. 2613–2622.), AdaptMatch (W. Huang, Y. Shi, Z. Xiong, and X. X. Zhu, "Adaptmatch: Adaptive matching for semisupervised binary segmentation of remote sensing images," IEEE Trans. Geosci. Remote Sensing, vol. 61, pp. 1–16, 2023.), ADVENT (T.-H. Vu, H. Jain, M. Bucher, M. Cord, and P. Prez, "Advent: Adversarial entropy minimization for domain adaptation in semantic segmentation," in Proc IEEE Comput Soc Conf Comput Vision Pattern Recognit, 2019, pp. 2512–2521.), and ST++ (L. Yang, W. Zhuo, L. Qi, Y. Shi, and Y. Gao, "St++: Make self-training work better for semi-supervised semantic segmentation," in Proc IEEE Comput Soc Conf Comput Vision Pattern Recognit, 2022, pp. 4268– 4277.) are compared.

[0063] (1) Experimental results of the Massachusetts dataset: Such as Figure 6As shown, in the first and second rows, when narrow roads are similar to the surrounding concrete background, other comparative models predict fragmented or even completely lost road information. In contrast, the method of this invention learns to extract global features from data branches with real labels through the DIMT module, helping the model achieve accurate road pixel classification in complex scenes where the foreground and background are highly similar. Furthermore, in road scenes with severe class imbalance, the method of this invention utilizes road prototype preservation and long-distance attention mechanisms to accurately extract dead ends outside the main road network. In the fourth and fifth rows, when roads are obscured by buildings and trees, the method of this invention can still achieve accurate road extraction. This is achieved through the RSM module, which learns lightweight global features through multi-directional sequential scanning to help accurately identify obscured pixels. In addition, the RPC module learns road prototype features from labeled data to enhance road features, assisting in the extraction of continuous road information for narrow and elongated roads while ensuring road continuity. In contrast, other models suffer from significant loss of road information due to their failure to fully utilize global information and insufficient consideration of class imbalance scenarios.

[0064] (2) Experimental results on the DeepGlobe dataset: like Figure 7 As shown, the second and third rows demonstrate the superiority of the method of the present invention in sandy terrain, where roads and backgrounds have similar spectral features. Other models can only produce fragmented extraction results, while the method of the present invention maintains continuity in the following ways: the DIMT module expands the learned feature space; the RPC module enhances road features in the case of class imbalance; and in densely built areas (the fifth and sixth rows), the dual mechanism of the RSM module overcomes the building occlusion problem.

[0065] The DIMT module further contributes by learning global features across scenes from real labeled data. For roads obscured by vegetation (first row), the global scanning attention of the RSM module and the directional feature enhancement of the RPC module achieve accurate extraction. It is worth noting that the visual performance of the fully supervised models (DLinkNet and TransRoadNet) is slightly better than that of the semi-supervised models (SemiRoadExNet and AdaptMatch). This may be because the fully supervised method avoids the propagation of errors in low-confidence regions in unlabeled data during training.

[0066] (3) Experimental results on the CHN6-CUG dataset: like Figure 8As shown in the second and fourth rows, when roads are obscured by the shadows of tall buildings, other models often lose key road information due to a lack of long-range dependencies and road feature enhancement. In contrast, the method of this invention enhances road features in complex environments by integrating local and global information from the image through the RSM module. Furthermore, the RPC module strengthens road feature learning in class-imbalanced scenarios by aligning with road features in labeled data. As can be seen from the first and fifth rows, when narrow roads are very similar to the surrounding flat ground, other models cannot accurately predict these roads. However, the DIMT module in this invention performs cross-domain global feature learning from labeled and unlabeled data and uses the RPC module to ensure the integrity and continuity of road features, especially for small target roads, thereby accurately predicting these roads.

[0067] In summary, this invention discloses a semi-supervised road extraction method based on feature calibration in optical remote sensing images. It aims to address the technical problems of existing fully supervised road extraction models relying on a large number of high-quality labeled samples, and semi-supervised models suffering from feature distribution mismatch, class imbalance leading to the omission of small roads, and edge prediction bias caused by occlusion. This invention constructs student and teacher sub-networks based on a Mean-Teacher semi-supervised framework. It introduces a DIMT module to achieve global feature distribution calibration between labeled and unlabeled data, utilizes an RPC module to alleviate class imbalance to preserve small road features, and designs an RSM module to simultaneously learn global and local features and uses gated fusion to optimize road edge prediction in occluded scenarios. Simultaneously, it achieves model training through a weighted combination of consistency loss, entropy minimization loss, and prototype loss. Experiments show that this invention achieves superior road extraction performance compared to existing fully supervised and semi-supervised models on datasets such as DeepGlobe and Massachusetts, significantly reducing labeling costs and handling road occlusion, small roads, and edge blurring issues in complex scenarios.

Claims

1. A semi-supervised road extraction method based on feature calibration from optical remote sensing images, characterized in that, include: The road extraction model is used to generate road extraction results from the optical remote sensing image to be processed. The road extraction model is obtained in the following way: Based on the Mean-Teacher semi-supervised framework, student subnetworks and teacher subnetworks with the same encoder-decoder network architecture are constructed. The student subnetwork and the teacher subnetwork are trained simultaneously using labeled and unlabeled data; the labeled data includes optical remote sensing images and their corresponding road segmentation binary maps, and the unlabeled data includes optical remote sensing images. After training, the teacher subnetwork is used as the road extraction model; In the bottleneck layer of the encoder-decoder network architecture, a DIMT module is provided to realize feature interaction and global feature complementarity between labeled and unlabeled data in the student subnetwork and the teacher subnetwork.

2. The semi-supervised optical remote sensing image road extraction method based on feature calibration according to claim 1, characterized in that, The DIMT module includes an interactive Mamba unit and an interactive Transformer unit; The interactive Mamba unit is used to flatten the features output by the encoder into a one-dimensional sequence along multiple spatial directions, and to perform feature interaction on the one-dimensional sequence. The interactive Transformer unit is used to perform cross-attention calculation on the features output by the interactive Mamba unit to achieve global feature complementarity.

3. The semi-supervised optical remote sensing image road extraction method based on feature calibration according to claim 2, characterized in that, The DIMT module is configured during the training phase as follows: Receive labeled and unlabeled data features output by the encoder; The labeled data features and unlabeled data features are sequentially passed through an interactive Mamba unit and an interactive Transformer unit to achieve feature interaction and global feature complementarity between labeled and unlabeled data. Configured during the inference phase as follows: Receive the optical remote sensing image features to be processed output by the encoder; The optical remote sensing image features to be processed are sequentially passed through an interactive Mamba unit and an interactive Transformer unit to enhance the features of the optical remote sensing image features to be processed.

4. The semi-supervised optical remote sensing image road extraction method based on feature calibration according to claim 3, characterized in that, In the interactive Mamba unit, the one-dimensional sequence is feature-interacted using the IS6 model; The IS6 model is configured during the training phase as follows: When performing selective state scanning on the one-dimensional sequence of the labeled data features, the one-dimensional sequence of the unlabeled data features is embedded; When performing selective state scanning on the one-dimensional sequence of unlabeled data features, the one-dimensional sequence of labeled data features is embedded, thereby achieving bidirectional feature interaction.

5. The semi-supervised optical remote sensing image road extraction method based on feature calibration according to claim 1, characterized in that, In the skip connections of the encoder-decoder network architecture, an RSM module for fusing global and local features is embedded, the RSM module including a global branch, a local branch and a gated fusion unit; The global branch uses the S6 model to extract global features; The local branches are extracted using multi-directional strip convolution to extract local features; The gated fusion unit is used to generate adaptive weights to fuse the global features and local features.

6. The semi-supervised optical remote sensing image road extraction method based on feature calibration according to claim 5, characterized in that, The RSM module is configured during the training phase as follows: Receive skip connection features from the encoder, the skip connection features including skip connection features of labeled data and skip connection features of unlabeled data; Global features of skip connection features are extracted through the global branch; Local features of skip connection features are extracted through the local branches; The gated fusion unit adaptively fuses the global and local features of the skip connection features and outputs the optimized features. Configured during the inference phase as follows: Receive skip connection features of the optical remote sensing image to be processed from the encoder; The global features of the skip connection features of the optical remote sensing image to be processed are extracted through the global branch; Local features of the skip connection features of the optical remote sensing image to be processed are extracted through the local branches; The gated fusion unit adaptively fuses the global and local features of the skip connection features of the optical remote sensing image to be processed, and outputs the optimized features.

7. The semi-supervised optical remote sensing image road extraction method based on feature calibration according to claim 1, characterized in that, Between the student subnetwork and the teacher subnetwork, an RPC module for feature calibration based on prototype features is set up; the RPC module includes an RPFB unit and a similarity calculation unit. The RPFB unit is used to store road prototype features from the labeled data extracted from the teacher subnetwork; The similarity calculation unit is used to calculate the similarity between the unlabeled data features in the student subnetwork and the road prototype features, and to calibrate the unlabeled data features in the student subnetwork based on the similarity.

8. The semi-supervised optical remote sensing image road extraction method based on feature calibration according to claim 1, characterized in that, The training process employs a combined loss function, which includes a consistency loss to constrain the difference between the prediction result and the supervision signal, an entropy minimization loss to improve the model's confidence in predicting unlabeled data, and a prototype loss to constrain the consistency between unlabeled data features and road prototype features.

9. A semi-supervised optical remote sensing image road extraction method based on feature calibration according to claim 8, characterized in that, The consistency loss is a weighted combination of Focal loss and Dice loss.

10. A semi-supervised optical remote sensing image road extraction method based on feature calibration, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1-9.

Images