Passive domain adaptive remote sensing image semantic segmentation method based on course learning guidance
The passive domain adaptive method guided by the course utilizes unbiased data augmentation and feature layer optimization to generate high-quality pseudo-labels, solving the problems of noise interference and high computational cost in semantic segmentation of remote sensing images, and improving segmentation accuracy and cross-domain adaptability.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SICHUAN NO 2 ELECTRIC POWER CONSTR CO
- Filing Date
- 2026-01-20
- Publication Date
- 2026-04-21
AI Technical Summary
Existing passive domain adaptive methods are susceptible to noise interference and have high computational costs in remote sensing image semantic segmentation, and cannot effectively adapt to resource-constrained environments.
By adopting a course-guided learning approach, through unbiased data augmentation, feature layer optimization, and output layer optimization, combined with densely connected decoders and adversarial loss, we optimize the model structure and parameter updates, generate high-quality pseudo-labels, reduce noise interference, and lower computational costs.
It improves the accuracy of semantic segmentation of remote sensing images and the cross-domain generalization ability of the model, reduces the dependence on computing resources, and is suitable for resource-constrained environments.
Smart Images
Figure CN121904375A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of remote sensing image processing technology, and in particular relates to a passive domain adaptive remote sensing image semantic segmentation method based on course learning guidance. Background Technology
[0002] With the continuous development of aerospace technology and Earth observation technology, the data sources for high-resolution remote sensing images are becoming increasingly abundant, making refined Earth observation possible. Semantic segmentation, with its pixel-level segmentation accuracy and mature model methods, has received widespread attention in the field of remote sensing image processing, and its segmentation results are widely used in many remote sensing fields such as land cover mapping, ecological environment protection, and urban planning.
[0003] However, the acquisition of remote sensing images is influenced by numerous factors, including geographical environment, imaging methods, and imaging conditions, leading to significant domain differences between different images. To reduce the feature distribution differences between cross-domain images and decrease the model's dependence on labels, unsupervised domain adaptation methods have been proposed. However, existing unsupervised domain adaptation methods are highly dependent on the accessibility of source domain data. In practical remote sensing applications, source domain data is often inaccessible due to privacy protection (national security, military secrets), data storage, transmission costs, and computational burdens. In such cases, traditional unsupervised domain adaptation methods cannot meet the requirements. Passive domain adaptation methods have been extensively studied, as they only need to consider the availability of the source pre-trained model and the unlabeled target domain data.
[0004] Currently, most advanced passive domain adaptive methods combine self-training mechanisms with feature cluster label refinement strategies, utilizing pre-trained source domain models to facilitate pseudo-label learning. They mainly face two challenges: 1. Feature cluster-based pseudo-label refinement strategies are susceptible to noise interference in the early stages of adaptation; 2. The distribution of clustering knowledge between adjacent samples requires a large memory repository, placing a significant burden on resource-constrained environments.
[0005] To address this challenge, this invention proposes a passive domain adaptive method suitable for semantic segmentation of high-resolution remote sensing images. This method can improve segmentation accuracy by generating high-quality pseudo-labels to guide the model's adaptive learning without accessing source domain data. Summary of the Invention
[0006] To solve the above-mentioned technical problems, the present invention is achieved through the following technical solution: This invention is a passive domain adaptive remote sensing image semantic segmentation method based on course-guided learning, comprising the following steps: S1. Pre-training and model initialization: Supervised training of the source domain model is performed using labeled source domain remote sensing image data to obtain a pre-trained source domain feature extractor and classifier. The source domain data will not be accessed again after training is completed in this stage. S2. Target Domain Model Construction: Based on the architecture of the pre-trained source domain model, a target domain semantic segmentation model using the average teacher framework is constructed. The target domain semantic segmentation model includes a student model and a teacher model. The teacher model is initialized using the weights of the pre-trained source domain model. S3. Unbiased data augmentation and multi-view input: Apply an unbiased data augmentation strategy to the unlabeled target domain remote sensing image to generate semantically consistent weakly augmented views and strongly augmented views; stack the weakly augmented views and strongly augmented views and input them into the teacher model for forward propagation; input the weakly augmented views into the student model. S4. Feature Layer Optimization: A dual optimization mechanism is introduced at the feature level to enable the model to learn feature representations with domain invariance and high intra-class consistency; S5. Output layer optimization: At the output layer, a hybrid supervision guided by the course learning strategy is adopted, and the source and intensity of the supervision signal are dynamically adjusted through the course learning strategy; S6. Model Structure Optimization and Policy Scheduling: The pacing factor from the course learning is used to optimize the model unfreezing and label generation process, while a densely connected decoder is designed to effectively fuse the multi-scale features of ground objects; S7. Model update mechanism: The teacher model does not perform backpropagation, and its parameters are updated through the exponential moving average of the student model parameters; S8. Model Training and Inference: Construct an overall objective function using joint adversarial loss, contrastive loss, and hybrid supervised loss, and iteratively optimize and update the parameters of the student model; input the remote sensing image of the target domain to be segmented into the trained student model to obtain pixel-level semantic segmentation results.
[0007] Furthermore, in step S1, the supervised training of the source domain model adopts the standard cross-entropy loss function, which calculates the log loss between the true class label and the network predicted probability.
[0008] Furthermore, in step S3, the implementation process of the unbiased data augmentation strategy is as follows: the original target domain image is randomly cropped and flipped to maintain the consistency of the basic spatial transformation, the processed image is backed up and input into different data processes respectively; the weak augmentation process only retains the results of the random cropping and flipping processing, while the strong augmentation process, on the basis of the random cropping and flipping, additionally uses color dithering, grayscale conversion, and Gaussian blur processing to simulate complex inter-domain changes.
[0009] Furthermore, the feature layer dual optimization mechanism in step S4 includes the following steps: Step S41, Domain Discriminative Adversarial Learning: Introduce a domain discriminative network to perform domain discrimination on the intermediate features generated by the teacher model and the student model, and calculate the adversarial loss. By confusing the domain discriminator, the feature extractor is prompted to generate a general feature representation with domain invariance. Step S42, Local Comparative Learning by Category: Embed a local comparative learning module for category-wise feature aggregation in the teacher model. By aggregating similar features and calculating the similarity across enhanced views, the distance between similar features is reduced and the distance between dissimilar features is increased.
[0010] Furthermore, in step S5, the pseudo-label selection process of the course learning strategy includes: calculating the prediction confidence of each pixel and normalizing the confidence of each category based on the category balance design; setting an initial pseudo-label selection ratio and linearly increasing the ratio at predetermined iteration intervals during training until a preset maximum value is reached; sorting the pixels under each category according to the normalized confidence, and determining the confidence threshold based on the current ratio to initially screen reliable pseudo-labels.
[0011] Furthermore, the course learning strategy includes a sample secondary correction process: based on the confidence threshold of the initial screening, the binary reliability score of the samples is calculated to divide the reliable sample group and the unreliable sample group; subsequently, by calculating the difference between the highest confidence and the second highest confidence in the prediction results of each pixel, samples with a difference greater than a predetermined threshold are pulled back from the unreliable sample group and added to the reliable sample group to increase the richness of reliable pseudo-labels.
[0012] Furthermore, in step S5, the soft labels or pseudo-labels are generated by weighted fusion of the prediction results of the teacher model and the student model; in the early stage of training, the prediction results of the teacher model are given higher weights, and as the training rounds increase, the weights gradually decrease according to a predefined decay function, while the weights of the prediction results of the student model increase accordingly, thereby realizing the course learning process that gradually transitions from teacher-led to student self-learning.
[0013] Furthermore, in step S6, the model unfreezing process specifically involves: the network weights of the teacher model being frozen in the early stages of training; as training progresses, the weights of each network layer are gradually unfrozen in order from lower layers closer to the input layer to higher layers closer to the output layer, according to a preset unfreezing priority, and in conjunction with a progress function based on the current training round, the total number of training rounds, and the course learning pace factor.
[0014] Further, in step S6, the densely connected decoder is implemented as follows: a one-to-one convolution is used to unify and simplify the number of channels in the feature maps from different depth network blocks of the encoder; in the decoding stage, a specific cross-scale layer is densely connected with the multi-scale feature maps obtained from all previous layers; the output features of each layer are obtained by fusing the feature information of all previous layers through a composite function consisting of separable convolution, linear rectified activation function and bilinear interpolation upsampling operation; finally, the outputs of each layer are upsampled and the classification probability map is obtained through a normalized exponential function.
[0015] Furthermore, in step S8, the overall objective function is a weighted sum of four losses: pseudo-label supervision loss, soft label supervision loss, contrastive loss, and adversarial loss, with the weight coefficients of each loss being preset configurable hyperparameters. After the model training is completed, the student model is used to perform forward propagation on the input target domain remote sensing image, outputting pixel-level semantic segmentation results.
[0016] The present invention has the following beneficial effects: 1. This invention proposes a self-training mechanism guided by course learning. By introducing a course learning strategy to assist self-training, it prioritizes learning from highly reliable pseudo-labels and then pulls back unreliable pseudo-labels to increase pseudo-label richness. Furthermore, it modifies and embeds the pacing factor in course learning to progressively guide the model's weight unfreezing process and label generation method, thereby mitigating the learning degradation caused by the large feature distribution difference between the two domains during cross-domain processes. This selective learning self-training strategy avoids the need for computationally expensive feature cluster label refinement frameworks that rely on repositories.
[0017] 2. This invention designs a non-offset data augmentation strategy and introduces contrastive representation learning on this basis to construct a local contrastive learning module for class-wise feature aggregation, which helps to alleviate the noise interference problem in the early stage of adaptation from the perspective of feature distribution.
[0018] 3. This invention features a dense multi-scale decoder designed to more effectively fuse multi-scale features of ground objects. Furthermore, a discriminant network is introduced to align teacher and student predictions along the feature dimension, effectively conveying the source domain feature distribution indirectly to the target domain and enhancing the model's cross-domain generalization ability.
[0019] Of course, any product implementing this invention does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description
[0020] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0021] Figure 1 This is a flowchart illustrating the passive domain adaptive remote sensing image semantic segmentation method based on course learning guidance of the present invention. Figure 2 This is an overall framework diagram of an embodiment of the present invention; Figure 3 This is a schematic diagram of the offset-free data enhancement process according to an embodiment of the present invention; Figure 4 The results are experimental visualizations of an embodiment of the present invention. Detailed Implementation
[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0023] Please see Figure 1-4 As shown, this invention is a passive domain adaptive remote sensing image semantic segmentation method based on course learning guidance, comprising the following steps: S1. Pre-training and Model Initialization: The source domain model is trained in a supervised manner using labeled source domain remote sensing image data to obtain a pre-trained source domain feature extractor and classifier. The source domain data is no longer accessed after training is completed in this stage. S2. Target Domain Model Construction: Based on the architecture of the pre-trained source domain model, a target domain semantic segmentation model using the average teacher framework is constructed. The target domain semantic segmentation model includes a student model and a teacher model. The teacher model is initialized using the weights of the pre-trained source domain model. S3. Unbiased Data Augmentation and Multi-View Input: Apply unbiased data augmentation strategies to unlabeled target domain remote sensing images to generate semantically consistent weak and strong augmented views; stack the weak and strong augmented views and input them into the teacher model for forward propagation, and input the weak augmented view into the student model. S4. Feature Layer Optimization: A dual optimization mechanism is introduced at the feature level to enable the model to learn feature representations with domain invariance and high intra-class consistency; S5. Output layer optimization: At the output layer, a hybrid supervision guided by the course learning strategy is adopted, and the source and intensity of the supervision signal are dynamically adjusted through the course learning strategy; S6. Model Structure Optimization and Policy Scheduling: The pacing factor from the course learning is used to optimize the model unfreezing and label generation process, while a densely connected decoder is designed to effectively fuse the multi-scale features of ground objects; S7. Model update mechanism: The teacher model does not perform backpropagation; its parameters are updated through the exponential moving average of the student model parameters. S8. Model Training and Inference: Construct the overall objective function using joint adversarial loss, contrastive loss, and hybrid supervised loss, and iteratively optimize and update the student model parameters; input the remote sensing image of the target domain to be segmented into the trained student model to obtain pixel-level semantic segmentation results.
[0024] In step S1, the supervised training of the source domain model uses the standard cross-entropy loss function, which calculates the log loss between the true class label and the network predicted probability.
[0025] In step S3, the implementation process of the unbiased data augmentation strategy is as follows: the original target domain image is randomly cropped and flipped to maintain the consistency of the basic spatial transformation. The processed image is backed up and input into different data processes respectively. The weak augmentation process only retains the results of random cropping and flipping. The strong augmentation process, on the basis of random cropping and flipping, additionally uses color dithering, grayscale conversion and Gaussian blur processing to simulate complex inter-domain changes.
[0026] The feature layer dual optimization mechanism in step S4 includes the following steps: Step S41, Domain Discriminative Adversarial Learning: Introduce a domain discriminative network to perform domain discrimination on the intermediate features generated by the teacher model and the student model, and calculate the adversarial loss. By confusing the domain discriminator, the feature extractor is prompted to generate a general feature representation with domain invariance. Step S42, Local Comparative Learning by Category: Embed a local comparative learning module for category-wise feature aggregation in the teacher model. By aggregating similar features and calculating the similarity across enhanced views, the distance between similar features is reduced and the distance between dissimilar features is increased.
[0027] In step S5, the pseudo-label selection process of the course learning strategy includes: calculating the prediction confidence of each pixel and normalizing the confidence of each category based on the category balance design; setting an initial pseudo-label selection ratio and linearly increasing the ratio at predetermined iteration intervals during training until a preset maximum value is reached; sorting the pixels under each category according to the normalized confidence, and determining the confidence threshold based on the current ratio to initially screen reliable pseudo-labels.
[0028] The course learning strategy includes a sample secondary correction process: based on the confidence threshold of the initial screening, the binary reliability score of the samples is calculated to divide the reliable sample group and the unreliable sample group; then, by calculating the difference between the highest confidence and the second highest confidence in the prediction results of each pixel, samples with a difference greater than a predetermined threshold are pulled back from the unreliable sample group and added to the reliable sample group to increase the richness of reliable pseudo-labels.
[0029] In step S5, soft labels or pseudo-labels are generated by weighted fusion of the prediction results of the teacher model and the student model. In the early stage of training, the prediction results of the teacher model are given higher weights. As the training rounds increase, the weights gradually decrease according to the predefined decay function, while the weights of the prediction results of the student model increase accordingly. This achieves a gradual transition from teacher-led to student-led learning in the course learning process.
[0030] In step S6, the model unfreezing process is as follows: the network weights of the teacher model are frozen in the early stage of training; as training progresses, the weights of each layer of the network are gradually unfrozen in the order from the lower layers closer to the input layer to the higher layers closer to the output layer, according to the preset unfreezing priority, and in combination with the progress function based on the current training round, the total training round, and the course learning pace factor.
[0031] In step S6, the densely connected decoder is implemented as follows: a one-to-one convolution is used to unify and simplify the number of channels in the feature maps from different depth network blocks of the encoder; in the decoding stage, a specific cross-scale layer is densely connected with the multi-scale feature maps obtained from all previous layers; the output features of each layer are obtained by fusing the feature information of all previous layers through a composite function consisting of separable convolution, linear rectified activation function and bilinear interpolation upsampling operation; finally, the outputs of each layer are upsampled and the classification probability map is obtained through a normalized exponential function.
[0032] In step S8, the overall objective function is the weighted sum of four losses: pseudo-label supervision loss, soft label supervision loss, contrastive loss, and adversarial loss. The weight coefficients of each loss are preset configurable hyperparameters. After the model is trained, the student model is used to perform forward propagation on the input target domain remote sensing image to output pixel-level semantic segmentation results.
[0033] One specific application of this embodiment is: This embodiment uses the LoveDA remote sensing dataset as the experimental data. It includes both urban and rural scenes, comprising 5987 remote sensing images with a spatial resolution of 0.3m. The training set contains 2522 images, the validation set 1699 images, and the test set 1796 images, covering seven categories including buildings, roads, agriculture, and backgrounds. In the LoveDA dataset, urban and rural scenes alternate between the source and target domains.
[0034] All experiments in this embodiment were trained on the PyTorch (1.10.0) framework, a single NVIDIA GTX3090 graphics card, an i9-10980XE CPU, and 128GB of RAM. Both the teacher-student networks used Deeplab-V2 based on ResNet50 and were pre-trained on ImageNet. In the experiments, the batch size for both pre-training and self-training was 8, with 4 weakly augmented images and 4 strongly augmented images in the self-training phase. This embodiment used SGD for optimization, with a learning rate of 10⁻⁴, a weight decay of 10⁻⁶, and a total of 20k training iterations, with 10k iterations each in the pre-training and self-training phases. In the self-training phase, pseudo-labels were generated and updated every 1k iterations, and performance evaluation was performed every 2k iterations. For data augmentation, this embodiment randomly cropped the original LoveDA images into 512×512 blocks and randomly rotated and mirrored them.
[0035] Implementation steps: Source domain model training: like Figure 2 As shown, to better illustrate the source-domain adaptive method, the following definition is required: the source domain that can access the real label. and source domain samples ; Target domain where the tag is unavailable and target domain samples , and These represent the number of samples in the source and target domains, respectively. Teacher Model Student model ,here and Represents the feature encoder. and Represents a semantic classifier. Indicates a strongly enhanced image. This represents a weakly enhanced image. During the adaptive preparation phase, the teacher and student models share the same weights. , The initial model weights for training the source domain data.
[0036] Labeled source domain images Input source domain model (including feature extractor) and semantic classifier ) Perform 10k iterations of training, using standard cross-entropy loss to supervise the source domain samples:
[0037] In the formula, To divide the total number of categories, Indicates the first The true category label for each pixel. The output of the segmentation network represents the first... Each pixel belongs to the category The predicted probability. Through loss construction, the model performs supervised learning on the source domain to obtain a source domain model with basic feature extraction capabilities, providing initial model weights for the subsequent domain adaptation stage.
[0038] Step 4, target domain model training: Offset-free data augmentation workflow design: Data augmentation is widely used to improve the generalization performance of models when faced with unknown data. This embodiment specifically focuses on several key factors affecting the cross-domain generalization performance of remote sensing, such as environmental conditions (lighting, climate), noise interference, shape and texture, and feature distribution, attempting to simulate different cross-domain distributions through data augmentation. Specifically, this embodiment designs a non-offset data augmentation process, such as... Figure 3 As shown. To ensure semantic consistency between strongly enhanced and weakly enhanced views, this embodiment first randomly crops and flips the original source domain image, and then backs up the processed images and inputs them into different data flows. During the strong enhancement process, conventional data augmentation strategies (such as color dithering, grayscale, and Gaussian blur) are used to create various visual features at the input level.
[0039] Course learning guidance self-training mechanism design: Traditional feature cluster-based label optimization methods are prone to introducing noise interference, leading to poor pseudo-label quality. This noise interference negatively impacts subsequent cluster structure, as the quality of the pseudo-labels themselves is crucial for generating high-quality clusters. Therefore, noise issues require special attention in traditional optimization methods. To address this, this embodiment designs a course-guided self-training mechanism. This mechanism follows an "easy-to-difficult" training pattern through self-paced course learning. The core idea is to generate pseudo-labels from high-confidence samples, aiming for most of them to be correct predictions. Once the model is updated and better adapted to the target domain, the model explores the remaining pseudo-labels with lower confidence. Here, this embodiment first describes the core part of course learning: the pseudo-label selection process. A class-balanced design is used to eliminate the influence of different confidence levels by normalizing the class-by-class confidence. The pseudo-label supervision loss on the target domain is defined as follows:
[0040]
[0041] In the formula, The number of images in the target domain. The number of pixels at a given spatial location in the image. To determine each category An independent parameter for the proportion of selected pseudo-labels. The differences between them introduce different levels of class bias into pseudo-label selection and resolve the inter-class balance problem. Regarding... The determination is made by predicting that Sort all pixels of the class by confidence and set the confidence level. Make Equivalent to in The probability value at this point, here Indicates the prediction is The number of pixels in each class. This strategy ranks pixels in each class separately. The probability is used as a reference for thresholding and confidence normalization.
[0042] about The calculation process is shown below, and the pseudo-label selection ratio in the algorithm process is as follows. The initial design uses 20%, increasing by 5% every k rounds based on experience, with a maximum percentage set at 50%. This differs from previous methods that directly utilize... Unlike the pseudo-label assignment method, this embodiment first calculates the binary reliability score using the sample's prediction confidence. :
[0043]
[0044] After obtaining the reliability score for each sample, this embodiment divides the entire sample space into reliable groups. and unreliable groups ,in , While this sample differentiation provides a good estimate of a reliable sample, There may be a lack of label richness. To address this, this embodiment introduces a new metric: the difference in Top-2 confidence scores. Some samples were pulled back to supplement the middle This serves as a potential remedy to increase the richness of pseudo-labels. Next, this embodiment fully considers the feature differences and complexities between different remote sensing images and designs a weighted joint pseudo-label generation strategy based on a teacher-student architecture. The idea is as follows: Since the teacher and student models are assigned the same initial weights and the teacher model's weights are updated progressively, the teacher model emphasizes the source domain distribution characteristics in the early stages of training, which can basically satisfy the transfer and generalization of simple categories or features. Therefore, in the early stages of training, a larger weight ratio is given to the teacher model. As training progresses, the learning difficulty increases, and the student model has basically mastered the feature distribution characteristics between the two domains. To avoid over-reliance on incorrect labels (generated from teacher predictions), it is necessary to gradually shift to the student model's self-learning. This strategy conforms to the principle of "easy first, difficult later" in course learning and can effectively avoid performance degradation caused by excessive domain differences, thereby improving the stability of training. The weighted joint pseudo-label generation formula is as follows:
[0045] In the formula, , The label pace coefficient represents the first step. Weights are assigned to pseudo-labels during rounds. The unfreezing strategy for the teacher model is as follows: Assume the teacher model has... The model has layers, and the weights of each layer are trainable. Initially, the weights of the teacher model are frozen. As training progresses, the weights of some layers are gradually unfrozen, guiding the student model step by step. The unfreezing progress of each layer is detailed below. It can be represented as:
[0046] In the formula, Indicates the first Layers in training rounds The degree of thawing at that time. It is a pacing factor that controls the rate of thawing. . It is a layer The unfreezing priority is as follows: lower-level layers (closer to the input layer) are unfrozen earlier, and higher-level layers (closer to the output layer) are unfrozen later.
[0047] Design of a Local Contrast Learning Module for Category-by-Category Feature Aggregation In addition to self-training, this embodiment also introduces unsupervised contrastive representation learning to construct a local contrastive learning module to further reduce the problem of noise propagation, such as... Figure 2 As shown. The specific implementation is as follows: First, the strongly augmented and weakly augmented views of the target domain are stacked and input into the teacher model to obtain domain features. Once pseudo-labels are generated, the pseudo-labels can be used to perform operations with the domain features to achieve category-specific region assignment. Regarding categories... The regional division strategy is expressed as follows:
[0048] The formula only shows a weakly enhanced view. The domain feature region representation, Similarly, using the spatial location mask described above, we can obtain the class-specific region representations of the two types of enhanced domain features. Next, we use average pooling to aggregate the domain features of different classes, as shown below:
[0049] In the formula, The aggregated features of each category domain, For category The number of pixels corresponding to the domain feature. This is achieved by considering each sample... Two aggregation domain features and Forcing similarity between them makes contrastive representation learning more meaningful, as expressed below:
[0050]
[0051] In the formula, For indicator functions, For temperature coefficient, The similarity is cosine. Previous SFDA methods mostly used label-dependent contrastive learning, while this embodiment focuses on label-free contrastive learning to minimize the interference of label noise.
[0052] Densely connected multi-scale decoder design: To effectively fuse multi-scale features from different layers, this embodiment proposes a densely connected multi-scale decoder, such as... Figure 2 As shown. The specific implementation is as follows: First, using Convolution preprocesses features from different blocks to obtain simplified features with equal channel numbers. Then, the specific cross-scale layer in the decoder is densely connected with all the previous features. Finally, the... The features of a layer are expressed as follows:
[0053] In the formula, It is a composite function consisting of separable convolution, ReLU function, and bilinear interpolation to improve model efficiency and transform previous features into specific scales suitable for subsequent fusion. Through the close interaction between multi-scale features, each part of the output is individually upsampled to the original spatial resolution and then normalized using the softmax function to obtain the classification probability. Compared with existing multi-scale decoders, the advantages of this embodiment are: (1) dense connections. Each scale feature can interact with all scale features in the previous layer, and these frequent interactions facilitate the extraction of complex multi-scale targets in remote sensing images. (2) lightweight operation. The use of separable convolution makes dense connections lightweight, thereby improving inference speed.
[0054] Loss function construction: For passive domain adaptive training, this embodiment, in addition to the pseudo-label supervised loss described above, and contrastive characterization loss In addition, soft-labeled supervision loss and adversarial loss are introduced. The former, referring to the design in Chapter 4, uses teacher predictions as soft labels to guide student model training. At the same time, it combines hard labels from the target domain to construct a hybrid supervision loss to improve the model's adaptability to complex remote sensing scenarios, thereby improving cross-domain generalization ability, as expressed below:
[0055] The latter introduces a discriminator. The intermediate features of the teacher-student model are used for adversarial training, prompting the model to learn more general intermediate representations. In this way, the model not only indirectly adjusts for the distribution differences between the source and target domains in the output layer, but also maintains domain independence in the intermediate features, increasing the model's transfer and generalization capabilities. The specific formula is as follows:
[0056] Finally, combining the pseudo-label supervision loss, soft-label supervision loss, contrastive representation loss, and adversarial loss, the overall loss function can be expressed as follows:
[0057] Experimental Analysis: Quantitative Analysis: The table below compares the segmentation accuracy and model efficiency of several existing passive domain adaptive methods and the proposed method. Analysis of the table shows that the proposed method in this embodiment performs best overall. Through auxiliary curriculum learning strategies and local contrastive representation learning, it can effectively and indirectly transfer source domain features without accessing the source domain data, while avoiding the limitations of traditional methods in label noise processing and spatial storage, thus possessing stronger domain generalization and practicality. Using ResNet50 as the backbone, it achieves mIoU improvements of 9.21% and 6.12% in rural® urban and urban® rural scenarios, respectively.
[0058]
[0059]
[0060] Qualitative analysis: Figure 4 The visualization results of LoveDA's cross-domain generalization are presented. In urban-to-rural tasks, the proposed method in this embodiment demonstrates higher accuracy and fewer misclassifications in the classification of water areas and wastelands. This is attributed to the local contrastive learning module that aggregates features by category, which effectively reduces early noise interference and improves the aggregation and distribution alignment of features. The dense multi-scale decoder also performs well, capable of fusing multi-scale features, especially outperforming other methods in the detailed segmentation of buildings and roads. Overall, the proposed method in this embodiment demonstrates strong stability and accuracy across multiple tasks.
[0061] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.
[0062] The preferred embodiments of the present invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention. The invention is limited only by the claims and their full scope and equivalents.
Claims
1. A passive domain adaptive remote sensing image semantic segmentation method based on course-guided learning, characterized by: Includes the following steps: Step S1: Pre-training and model initialization: Supervised training of the source domain model is performed using labeled source domain remote sensing image data to obtain a pre-trained source domain feature extractor and classifier. The source domain data will not be accessed again after training is completed in this stage. Step S2: Target domain model construction: Based on the architecture of the pre-trained source domain model, construct a target domain semantic segmentation model using the average teacher framework. The target domain semantic segmentation model includes a student model and a teacher model. Initialize the teacher model using the weights of the pre-trained source domain model. Step S3: Unbiased data augmentation and multi-view input: Apply unbiased data augmentation strategies to the unlabeled target domain remote sensing image to generate semantically consistent weakly and strongly augmented views; The weakly enhanced view and the strongly enhanced view are stacked and then input into the teacher model for forward propagation, and the weakly enhanced view is input into the student model; Step S4: Feature layer optimization: A dual optimization mechanism is introduced at the feature level to enable the model to learn feature representations with domain invariance and high intra-class consistency; Step S5: Output layer optimization: At the output layer, a hybrid supervision guided by the course learning strategy is adopted, and the source and intensity of the supervision signal are dynamically adjusted through the course learning strategy; Step S6: Model structure optimization and policy scheduling: The pacing factor learned in the course is used to optimize the model unfreezing and label generation process, while a densely connected decoder is designed to effectively fuse the multi-scale features of ground objects; Step S7: Model update mechanism: The teacher model does not perform backpropagation, and its parameters are updated through the exponential moving average of the student model parameters; Step S8: Model Training and Inference: Construct the overall objective function using joint adversarial loss, contrastive loss, and hybrid supervised loss, and iteratively optimize and update the student model parameters; input the remote sensing image of the target domain to be segmented into the trained student model to obtain pixel-level semantic segmentation results.
2. The passive domain adaptive remote sensing image semantic segmentation method based on course learning guidance according to claim 1, characterized in that, In step S1, the supervised training of the source domain model uses the standard cross-entropy loss function, which calculates the log loss between the true class label and the network predicted probability.
3. The passive domain adaptive remote sensing image semantic segmentation method based on course learning guidance according to claim 1, characterized in that, In step S3, the implementation process of the unbiased data augmentation strategy is as follows: the original target domain image is randomly cropped and flipped to maintain the consistency of the basic spatial transformation, and the processed image is backed up and input into different data processes respectively. The weak enhancement process only retains the results of the random cropping and flipping, while the strong enhancement process, in addition to the random cropping and flipping, uses color dithering, grayscale conversion, and Gaussian blurring to simulate complex inter-domain changes.
4. The passive domain adaptive remote sensing image semantic segmentation method based on course-guided learning as described in claim 1, characterized in that, The feature layer dual optimization mechanism in step S4 includes the following steps: Step S41, Domain Discriminative Adversarial Learning: Introduce a domain discriminative network to perform domain discrimination on the intermediate features generated by the teacher model and the student model, and calculate the adversarial loss. By confusing the domain discriminator, the feature extractor is prompted to generate a general feature representation with domain invariance. Step S42, Local Comparative Learning by Category: Embed a local comparative learning module for category-wise feature aggregation in the teacher model. By aggregating similar features and calculating the similarity across enhanced views, the distance between similar features is reduced and the distance between dissimilar features is increased.
5. The passive domain adaptive remote sensing image semantic segmentation method based on course learning guidance according to claim 1, characterized in that, In step S5, the pseudo-label selection process of the course learning strategy includes: calculating the prediction confidence of each pixel and normalizing the confidence of each category based on the category balance design; setting an initial pseudo-label selection ratio and linearly increasing the ratio at predetermined iteration intervals during training until a preset maximum value is reached; sorting the pixels under each category according to the normalized confidence, and determining the confidence threshold based on the current ratio to initially screen reliable pseudo-labels.
6. The passive domain adaptive remote sensing image semantic segmentation method based on course learning guidance according to claim 5, characterized in that, The course learning strategy includes a sample secondary correction process: based on the confidence threshold of the initial screening, the binary reliability score of the samples is calculated to divide the reliable sample group and the unreliable sample group. Subsequently, by calculating the difference between the highest confidence level and the second highest confidence level in the prediction results of each pixel, samples with a difference greater than a predetermined threshold are pulled back from the unreliable sample group and added to the reliable sample group.
7. The passive domain adaptive remote sensing image semantic segmentation method based on course learning guidance according to claim 5, characterized in that, In step S5, the soft label or pseudo label is generated by weighted fusion of the prediction results of the teacher model and the student model; in the early stage of training, the prediction results of the teacher model are given higher weights, and as the training rounds increase, the weights gradually decrease according to a predefined decay function, while the weights of the prediction results of the student model increase accordingly.
8. The passive domain adaptive remote sensing image semantic segmentation method based on course learning guidance according to claim 1, characterized in that, In step S6, the model unfreezing process is as follows: the network weights of the teacher model are frozen in the early stage of training; as training progresses, the weights of each layer of the network are gradually unfrozen in the order from the lower layers closer to the input layer to the higher layers closer to the output layer, according to the preset unfreezing priority, and in combination with the progress function based on the current training round, the total training round, and the course learning pace factor.
9. The passive domain adaptive remote sensing image semantic segmentation method based on course learning guidance according to claim 8, characterized in that, In step S6, the densely connected decoder is implemented as follows: a one-to-one convolution is used to unify and simplify the number of channels in the feature maps from different depth network blocks of the encoder; in the decoding stage, a specific cross-scale layer is densely connected with the multi-scale feature maps obtained from all previous layers; the output features of each layer are obtained by fusing the feature information of all previous layers through a composite function consisting of separable convolution, linear rectified activation function and bilinear interpolation upsampling operation; finally, the outputs of each layer are upsampled and the classification probability map is obtained through a normalized exponential function.
10. The passive domain adaptive remote sensing image semantic segmentation method based on course learning guidance according to claim 1, characterized in that, In step S8, the overall objective function is a weighted sum of four losses: pseudo-label supervision loss, soft label supervision loss, contrastive loss, and adversarial loss. The weight coefficients of each loss are preset configurable hyperparameters. After the model training is completed, the student model is used to perform forward propagation on the input target domain remote sensing image to output pixel-level semantic segmentation results.