An unsupervised domain adaptation image semantic segmentation method based on task decomposition

Abstract

The application discloses a task-decomposition-based unsupervised domain adaptive image semantic segmentation method, and establishes a task-decomposition-based unsupervised domain adaptive image semantic segmentation model, which comprises the following steps: extracting a part of shallow network layers from a feature extractor as a shallow target domain feature extractor, and meanwhile, constructing a twin shallow feature extractor with the same structure as the target domain feature extractor; the remaining network layers of the feature extractor are used as a shared deep feature extractor; after source domain images and target domain images pass through the source domain feature extractor and the target domain feature extractor, source domain shallow features and target domain shallow features are obtained; then, the source domain shallow features and the target domain shallow features pass through the deep feature extractor to obtain source domain deep features and target domain deep features; and finally, the source domain deep features and the target domain deep features are sent into a segmentation network to obtain source domain prediction results and target domain prediction results. The two twin shallow feature extractors are used to undertake domain alignment and semantic segmentation tasks, the two tasks are decomposed from a single main network, and the semantic segmentation of the target domain is prevented from being disturbed by the source domain supervised training.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and in particular relates to an unsupervised domain adaptive image semantic segmentation method based on task deconstruction. Background Technology

[0002] Image semantic segmentation is a fundamental computer vision task. In recent years, thanks to the rapid development of deep learning technology, numerous works related to image semantic segmentation have emerged, achieving great success in fields such as autonomous driving, intelligent monitoring, and medical image analysis. However, the success of these works typically relies on large amounts of labeled data for model training, and the high cost of annotation limits the usability of these methods in real-world applications. To address this issue, researchers have proposed the Unsupervised Domain Adaptation (UDA) algorithm, which aims to train a cross-domain segmentation network using a labeled dataset related to the target task (referred to as the source domain) and an unlabeled dataset on the target task (referred to as the target domain), thereby achieving considerable segmentation performance without requiring any annotation of the target domain data. In UDA-based image semantic segmentation research, domain adversarial techniques have received widespread attention due to their powerful distribution alignment capabilities and excellent performance.

[0003] Although the existing methods mentioned above effectively alleviate the domain margin problem in UDA-based cross-domain image semantic segmentation, there are still two potential problems caused by model design that limit the improvement of cross-domain segmentation performance.

[0004] One problem is the overly strong coupling between tasks. Current work essentially performs domain alignment and semantic segmentation tasks simultaneously on the same backbone. This structure often affects the network model, especially the feature extraction part, causing it to focus on different tasks and preventing it from concentrating on any one task. For example, excessive focus on domain alignment can cause the feature extractor to neglect learning enough knowledge for semantic segmentation. Conversely, excessive focus on semantic segmentation can weaken the feature extractor's ability to perform domain alignment. This overly coupled and competitive relationship between tasks prevents each task from being fully learned, thus limiting the network's performance.

[0005] The second problem is the network's overfitting to the source domain. To enable the network model to have sufficient semantic segmentation capabilities, it is usually trained using source domain data and its labels. Therefore, the model fits the distribution of the source domain data very well. However, since both the source and target domain data are extracted using the same feature extractor, the model's good fit to the source domain can interfere with the learning of the target domain, which lacks strong supervision signals. This causes the domain alignment results to be more biased towards the source domain distribution, thus affecting the network's performance in the target domain. Summary of the Invention

[0006] To address the aforementioned issues, this invention proposes an unsupervised domain adaptive image semantic segmentation method based on task deconstruction. This method utilizes two twin shallow feature extractors to undertake the domain alignment task and the semantic segmentation task, respectively, thereby deconstructing these two tasks from a single backbone network and avoiding interference from source domain supervised training in the semantic segmentation of the target domain.

[0007] To achieve the above objectives, the technical solution adopted by this invention is: an unsupervised domain adaptive image semantic segmentation method based on task deconstruction, comprising the following steps:

[0008] Establish an unsupervised domain adaptive image semantic segmentation model based on task deconstruction, including: extracting a portion of shallow network layers from the feature extractor E as a shallow target domain feature extractor e. tgt At the same time, construct a connection with e tgt The same structure twin shallow feature extractor e src Used to extract shallow features from the source domain; the remaining network layers of feature extractor E serve as a shared deep feature extractor e. share Connected to e tgt With e src ;

[0009] An unsupervised adaptive image semantic segmentation method based on task deconstruction includes the following steps:

[0010] S10, given the source domain input image χ S and the target domain input image χ T , respectively through e src and e tgt Then, the shallow features ε of the source domain are obtained. S and shallow features ε of the target domain T ;

[0011] S20, ε S and ε T After e share Then, the deep features f of the source domain are obtained respectively. S and deep features f of the target domain T ;

[0012] S30, f S and f T After being fed into the segmentation network H, the prediction results p of the source and target domains are obtained. S and p T .

[0013] Furthermore, the prediction result p in the source domain S The above uses cross-entropy loss Lce To train the source domain segmentation network This indicates the connection between modules.

[0014] Furthermore, the domain alignment task is deconstructed into e src Above, by adding a discriminator D located in a shallow layer shall This causes the shallow feature distribution of the source domain to move closer to the target domain, and the loss is calculated as follows:

[0015]

[0016] in Discriminator D shall The output is the number of all elements at a given spatial location.

[0017] Furthermore, a domain discriminator D was added at the output level. out Adversarial loss is used to further reduce the domain margin between the source and target domains. The calculation formula is as follows:

[0018]

[0019] in, Discriminator D out The output is the number of all elements at a given spatial location.

[0020] Furthermore, the discriminator D is established. shall and D out The discriminant losses are as follows:

[0021]

[0022]

[0023] Furthermore, a consistency learning module is established, introducing another segmentation head with the same structure but randomly initialized. To get f T Another segmentation prediction result And through consistency loss constraint p T and Consistency.

[0024] Furthermore, the consistency loss is calculated using the 2-norm, as shown in the formula:

[0025]

[0026] Furthermore, uncertainty estimation is introduced into the consistency loss constraint to guide the segmentation model to focus more on learning reliable regions.

[0027] Furthermore, when introducing uncertainty estimation into the consistency loss constraint to guide the segmentation model to focus more on learning reliable regions:

[0028] Regarding the prediction result p T The information entropy method is used to calculate the corresponding uncertain graph U;

[0029] The prediction result p is obtained through (1-U). T The confidence graph shows that the closer the value is to 1, the higher the confidence of the network's prediction for that location, and vice versa.

[0030] Guided by the confidence graph, update the consistency loss.

[0031] Furthermore, the uncertain value of the i-th element in U can be obtained by the following formula:

[0032]

[0033] in, p T For the predicted value of the c-th category at position i, the log function ensures U with the total number of categories C as the base. i The value ranges from 0 to 1; U i This reflects the model's prediction results. The degree of uncertainty is such that the closer it is to 0, the more certain the network is, and vice versa.

[0034] Guided by the confidence plot, the consistency loss is updated as follows: Where ⊙ represents element-wise multiplication;

[0035] Therefore, the total loss for training the entire segmentation model is: λ1, λ2, and λ3 are used to balance the contributions of each loss term.

[0036] The beneficial effects of adopting this technical solution are:

[0037] This invention deconstructs the domain alignment and semantic segmentation tasks by designing two twin shallow feature extractors. Specifically, these two shallow feature extractors extract shallow features from the source and target domain data, respectively. Considering that the differences between the source and target domains mainly lie in shallow style information, adversarial learning is employed to make the distribution of features extracted by the source domain shallow feature extractor closer to that of the target domain. The target domain shallow feature extractor is only responsible for sending the features to the subsequent network for semantic segmentation. Through this structural design, the domain alignment and semantic segmentation tasks are deconstructed onto different shallow feature extractors, thereby reducing the risk of mutual interference.

[0038] This invention utilizes source domain data for supervised training only in the shallow feature extractor of the source domain and in subsequent segmentation networks, while the shallow feature extractor of the target domain is invisible to the source domain data. Furthermore, since the shallow feature extractor of the source domain undertakes the task of domain adaptation, the subsequent segmentation task in the target domain will also be performed on domain-aligned features. Therefore, the influence of the network's fitting of the source domain data on the semantic segmentation of the target domain data is greatly reduced, improving the segmentation performance in the target domain.

[0039] To learn more discriminative target domain features, this invention designs a consistency learning module for the target domain data. Specifically, this module includes two structurally identical but differently initialized segmentation heads that segment the same target domain data and constrain their output segmentation predictions to be consistent. Inspired by the low-density separation hypothesis, the proposed consistency learning module helps the model learn more robust and discriminative features. Furthermore, this method introduces uncertainty estimation to guide the consistency learning to focus more on regions with high confidence in the segmentation predictions, reducing the interference of unreliable regions on the learning of the target domain data. Attached Figure Description

[0040] Figure 1 This is a schematic diagram of the process of an unsupervised domain adaptive image semantic segmentation method based on task deconstruction according to the present invention;

[0041] Figure 2 This is a visualization of the ablation experiment results in an embodiment of the present invention;

[0042] Figure 3 This is a visualization of the experimental results of the feature distribution of the source domain and the target domain in an embodiment of the present invention;

[0043] Figure 4 This is a visualization of the experimental results of the category feature distribution in the target domain in an embodiment of the present invention. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described below with reference to the accompanying drawings.

[0045] In this embodiment, see Figure 1 As shown, this invention proposes an unsupervised domain adaptive image semantic segmentation method based on task deconstruction.

[0046] Establish an unsupervised domain adaptive image semantic segmentation model based on task deconstruction, including: extracting a portion of shallow network layers from the feature extractor E as a shallow target domain feature extractor e. tgt At the same time, construct a connection with e tgt The same structure twin shallow feature extractor e srcUsed to extract shallow features from the source domain; the remaining network layers of feature extractor E serve as a shared deep feature extractor e. share Connected to e tgt With e src .

[0047] An unsupervised adaptive image semantic segmentation method based on task deconstruction includes the following steps:

[0048] S10, given the source domain input image χ S and the target domain input image χ T , respectively through e src and e tgt Then, the shallow features ε of the source domain are obtained. S and shallow features ε of the target domain T ;

[0049] S20, ε S and ε T After e share Then, the deep features f of the source domain are obtained respectively. S and deep features f of the target domain T ;

[0050] S30, f S and f T After being fed into the segmentation network H, the prediction results p of the source and target domains are obtained. S and p T .

[0051] As an optimization of the above embodiments, the prediction result p in the source domain... S The above uses cross-entropy loss L ce To train the source domain segmentation network This indicates the connection between modules.

[0052] Deconstruct the domain alignment task into e src Above, by adding a discriminator D located in a shallow layer shall This causes the shallow feature distribution of the source domain to move closer to the target domain, and the loss is calculated as follows:

[0053]

[0054] in Discriminator D shall The output is the number of all elements at a given spatial location.

[0055] The reason why e can be used src Extracted shallow features ε SDomain alignment is performed because the domain margin between the source and target domains is mainly reflected in low-dimensional information such as color, brightness, texture, and contrast. Shallow CNN networks naturally possess the ability to extract this low-dimensional information; therefore, this invention allows e src It is responsible for field alignment. After field alignment, The knowledge learned from the source domain data can be considered as height-domain invariant knowledge, which can be transferred to the learning of the target domain.

[0056] Although the source and target domains are aligned on shallow features, there may still be small differences between the two domains in high-dimensional space, causing a mismatch in the output distribution. A domain discriminator D is added at the output level. out Adversarial loss is used to further reduce the domain margin between the source and target domains. The calculation formula is as follows:

[0057]

[0058] in, Discriminator D out The output is the number of all elements at a given spatial location;

[0059] Because in the shallow layer of the network, The domain alignment strategy adopted is to pull the distribution of the source domain toward the target domain. In order to ensure more comprehensive domain alignment, we consider pulling the distribution of the target domain toward the source domain.

[0060] The discriminator D shall and D out The discriminator D is enhanced by using a discriminative loss to correctly identify whether features originate from the source or target domain during adversarial learning. shall and D out The discriminant losses are as follows:

[0061]

[0062]

[0063] Domain alignment tasks are performed solely by e src Bearing responsibility, while the target domain branch The domain alignment task no longer needs to be given much attention; the focus can be solely on the subsequent segmentation task in the target domain. Through this structural design, the present invention deconstructs the domain alignment and segmentation tasks, reducing mutual interference during their learning process and allowing them ample learning opportunities. Furthermore, the training process only requires the source domain branch... Supervised training is performed on the source domain data, while the target domain branch... Therefore, there's no need to focus on performance in the source domain, and it won't be negatively affected by overfitting of the segmentation network in the source domain. After training, only the inference phase retains... As the final model, the other modules, including e src D shall and D out If it is, then discard it directly.

[0064] As an optimization of the above embodiments, but for the target domain branch In contrast, its loss function has only one adversarial loss at the output level. The lack of an effective semantic-aware loss function for the target domain may lead to e tgt The optimal parameters cannot be learned, and e share Extracted features f T The method lacks discriminative power in the target domain. To compensate for this deficiency, this method establishes a consistency learning module and introduces another segmentation head with the same structure but randomly initialized. To get f T Another segmentation prediction result And through consistency loss constraint p T and Consistency.

[0065] The consistency loss is calculated using the 2-norm, and the formula is as follows:

[0066]

[0067] Due to H and These are two networks with the same structure but different initial weights. The introduction of f can be seen as... T Perturbations were added to the network layers during the learning process. The segmentation prediction results p under different perturbations were constrained. T and The consistency of data can theoretically help segmentation networks better discover the distribution and structure of these unlabeled data and learn more robust and discriminative features from them.

[0068] Considering the lack of labeled supervision information in the target domain data, the prediction results of the segmentation model may not be reliable. This invention further introduces uncertainty estimation into the consistency loss constraint to guide the segmentation model to focus more on learning reliable regions. Specifically:

[0069] Regarding the prediction result p T The information entropy method is used to calculate the corresponding uncertain graph U;

[0070] The uncertain value of the i-th element in U can be obtained by the following formula:

[0071]

[0072] in, p T For the predicted value of the c-th category at position i, the log function ensures U with the total number of categories C as the base. i The value ranges from 0 to 1; U i This reflects the model's prediction results. The degree of uncertainty is such that the closer it is to 0, the more certain the network is, and vice versa.

[0073] The prediction result p is obtained through (1-U). T The confidence graph shows that the closer the value is to 1, the higher the confidence of the network's prediction for that location, and vice versa.

[0074] Guided by the confidence plot, the consistency loss is updated as follows: Where ⊙ represents element-wise multiplication;

[0075] Therefore, the total loss for training the entire segmentation model is: λ1, λ2, and λ3 are used to balance the contributions of each loss term.

[0076] This invention will be validated on a domain adaptive segmentation task (GTA5→Cityscapes and SYNTHIA→Cityscapes) from two synthetic datasets, GTA5 and SYNTHIA, to the real dataset Cityscapes. Experimental results will be measured using the mIoU metric, which is commonly used in segmentation tasks.

[0077] To evaluate the performance superiority of the proposed method, this section compares it with the following popular UDA-based cross-domain image semantic segmentation methods: 1) methods without self-training, including AdaptSeg, CLAN, and Advent; 2) methods with self-training, including AdaptPatch, Label-Driven, FADA, LTI, FDA, TPLD, and MetaCorrection. Only the first stage is considered, where AdaptSeg, CLAN, Advent, AdaptPatch, and FADA employ adversarial learning-based domain alignment, Label-Driven, LTI, and FDA employ style transfer-based domain alignment, and TPLD and MetaCorrection employ self-training-based domain alignment. Experimental results for all compared methods are derived from their respective original papers. Segmentation performance on a single class is measured using IoU, and the overall segmentation performance across all classes is measured using mIoU.

[0078] Table 1 Comparison of results from different segmentation methods on GTA5 → Cityscapes.

[0079] ST stands for self-training. The best result is indicated in bold, and the second-best result is indicated in underline.

[0080]

[0081]

[0082] Table 1 presents the comparative results on the GTA5 → Cityscapes task, covering the individual IoU scores and the overall mIoU score for all 19 common classes. "SourceOnly" represents training the segmentation network using only source domain data and then testing it on the target domain data. It can be observed that this method without domain adaptation achieves the worst segmentation performance, with IoU scores of only 27.0% and 53.9% for easily identifiable categories like "road" and "building," and an mIoU score of only 36.0%. In contrast, the domain-adaptive method improves segmentation performance across all categories, with the mIoU improvement reaching up to 16%. Among all domain-adaptive methods, the self-training (ST) method generally outperforms the non-self-training method in segmentation performance. This is because the self-training strategy fully exploits the semantic information of the target domain through pseudo-labeling. Furthermore, among all self-trained domain adaptation methods, TPLD and MetaCorrection exhibit highly competitive segmentation performance, even achieving the highest IoU scores across multiple classes, with mIoU scores reaching 51.2% and 52.1%, respectively. Compared to these two methods, our proposed method (Ours) further improves the mIoU score by 1.7% and 0.8%, and its IoU scores rank among the top in most classes, demonstrating its superiority.

[0083] Table 2 compares the results of different segmentation methods on SYNTHIA→Cityscapes. ST indicates self-training. The best result is indicated in bold, and the second best result is indicated in underline.

[0084]

[0085]

[0086] Table 2 presents the comparative results on the SYNTHIA→Cityscapes task, covering the individual IoU scores and the overall mIoU score across all 13 common classes. The results show that the self-trained method significantly outperforms the non-self-trained method, achieving an average performance improvement of approximately 6% in mIoU. Furthermore, among all the self-trained comparative methods, Label-Driven and TPLD achieved mIoU scores of 53.1% and 53.5%, respectively, outperforming other comparative methods. Our proposed method not only achieved the highest mIoU score of 54.9%, but also achieved the best and second-best IoU scores in most classes, further demonstrating its superiority over other comparative methods and its adaptability to various cross-domain segmentation tasks.

[0087] The effectiveness of key strategies or modules of the proposed method is also ablated, including task deconstruction strategies and shallow domain adversarial losses. Domain adversarial loss in output space Consistent learning loss L con Uncertainty Estimation (UE) strategy. All ablation experiments were conducted on the GTA5→Cityscapes task. Table 3 shows the experimental results of the first stage without self-training. "w / o" indicates that the corresponding module was removed from the proposed method. It is worth noting that w / oL con This means removing the entire consistency learning module, including the UE, while w / oUE means removing only the uncertainty estimation strategy from the consistency learning module, retaining the consistency loss on the target domain data.

[0088] Table 3. Ablation experiment results without self-training on GTA5→Cityscapes

[0089]

[0090] AdaptSeg is a representative work that integrates domain adaptation and semantic segmentation tasks into a single backbone network, while w / oL con After removing the consistency learning module, compared to AdaptSeg which only retains the structure of two twin shallow feature extractions, the results of the two methods can be compared to verify the effectiveness of the task deconstruction strategy proposed in this paper. Table 3 shows that w / oL con The proposed task deconstruction strategy outperforms AdaptSeg in segmentation across most categories, improving IoU by 6.3% and 10.6% on person and car, respectively, demonstrating that it enhances the model's segmentation performance in the target domain. This is further demonstrated by comparing w / o... Ours can observe that removing shallow domain adversarial loss Subsequently, the model's segmentation performance in the target domain showed a significant decline, with the mIoU score decreasing by 2.4%, demonstrating... Contributions made to domain alignment. In contrast, the domain adversarial loss of the output space. The contribution of the variable is much smaller; after removal, the mIoU score only decreased by 0.9%, and the model's segmentation performance remained among the best in a considerable number of categories, confirming that the domain adversarial loss in the output space described in Section 3.3.2 only plays a supporting role. Finally, compared to Ours, w / oUE performance decreased in most categories. Especially in categories with a large pixel proportion, such as road and sidewalk, the performance drop was as high as 15% or more, demonstrating that the proposed uncertainty estimation strategy is crucial in guiding the model to learn reliable knowledge.

[0091] Figure 2 The visualization results of the ablation experiments are presented. As can be seen from the figures, SourceOnly performs poorly in segmenting larger target classes such as road, sidewalk, and truck. Although AdaptSeg shows a slight performance improvement, it still has many missegments in pole, truck, and bicycle. Furthermore, compared to the method proposed in this invention, improvements are achieved by removing the task deconstruction strategy and shallow domain adversarial loss. Domain adversarial loss in output space Consistent learning loss L con After implementing the uncertainty estimation strategy, the quality of the segmentation results output by the model decreased, with numerous missegments occurring in the building, truck, and vegetable domains. In particular, the quality of the segmentation results in the target domain showed the most significant decline after removing the uncertainty estimation strategy. These visualizations qualitatively demonstrate the importance of the proposed strategy or module for cross-domain semantic segmentation.

[0092] To verify the impact of the proposed task deconstruction strategy on domain adaptation, the t-distributed random nearest neighbor embedding technique was used to analyze AdaptSeg and w / oL in the ablation experiment. con The feature distributions of the source and target domains extracted by the model were visualized, and the results are as follows: Figure 3 As shown in the diagram. Source domain features are represented in dark color, and target domain features are represented in light color. It is worth noting that, compared to AdaptSeg, w / oL con It only adopted more of the task structure strategies proposed in this chapter. Through observation... Figure 4 It can be clearly observed that, compared to AdaptSeg, the method in this chapter achieves more thorough alignment of the feature distributions between the source and target domains. These experimental results demonstrate that the task deconstruction strategy proposed in this chapter effectively promotes domain alignment between the source and target domains, improving the model's segmentation performance in the target domain.

[0093] This invention extracts the shallow portion of a single segmentation backbone network and constructs two twin shallow feature extractors through replication to extract shallow features from the source and target domains, respectively. Considering that the main difference between the source and target domains lies in the low-dimensional style information of the image, this method applies adversarial learning to make the shallow features of the source domain converge towards the target domain, thus achieving domain alignment. The shallow features of the target domain are then directly fed into the subsequent deep feature extraction and segmentation networks for semantic segmentation. In this way, the domain alignment and segmentation tasks are deconstructed into the shallow feature extractors of the source and target domains, respectively. Furthermore, to fully exploit the discriminative information in the target domain that can be used for segmentation, this invention proposes a consistency learning module, aiming to constrain the output consistency of the target domain data in segmentation networks with different initializations, thereby improving the robustness and learning ability of the network. Simultaneously, considering the unreliability of the network's predictions for unlabeled data, this method introduces uncertainty estimation to weight the consistency learning, guiding the network to learn knowledge from more reliable predictions.

[0094] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.

Claims

1. An unsupervised domain adaptation image semantic segmentation method based on task decomposition, characterized in that, The method comprises the steps of: Establish an unsupervised domain adaptation image semantic segmentation model based on task decomposition, including: extracting a part of the shallow network layer from the feature extractor E as a shallow target domain feature extractor e tgt , and constructing a twin shallow feature extractor e tgt with the same structure as e src for extracting the shallow features of the source domain; the remaining network layers of the feature extractor E as a shared deep feature extractor e share connecting e tgt and e src ; An unsupervised domain adaptation image semantic segmentation method based on a task decomposition unsupervised domain adaptation image semantic segmentation model, comprising the steps of: S10, input image x of given source domain S and input image x of target domain T , respectively, after e src and e tgt , source domain shallow feature ε S and target domain shallow feature ε T are obtained; S20, ε S and ε T After e share , the deep features f S of the source domain and the deep features f T of the target domain are obtained respectively. S30, f S and f T The prediction results p of the source domain and the target domain are obtained after being sent into the segmentation network H S and p T ; A consistency learning module is established, and another segmentation head with the same structure but random initialization is introduced to obtain f T Another segmentation prediction result of f And the consistency of p T With Consistency constraint; In the consistency loss constraint, the uncertainty estimation is introduced to guide the segmentation model to pay more attention to the learning of reliable areas: For the prediction result p T , the information entropy is used to calculate the corresponding uncertainty graph U; The confidence map for the prediction result p is obtained by (1-U) T The value in the map is closer to 1, the higher the confidence of the network for the prediction result of the position, and vice versa. Under the guidance of the confidence map, the consistency loss is updated.

2. The unsupervised domain adaptation image semantic segmentation method based on task decomposition according to claim 1, characterized in that, The prediction result p of the source domain S The cross-entropy loss L is used to train the segmentation network of the source domain ce The cross-entropy loss L is used to train the segmentation network of the source domain Indicates the connection between the modules.

3. The task deconstruction based unsupervised domain adaptation image semantic segmentation method according to claim 2, characterized in that, The domain alignment task is decomposed into e src On the other hand, by adding a shallow-level discriminator D shall So that the shallow-level feature distribution of the source domain is close to the target domain, and the loss is calculated as: wherein represents the discriminator D shall the number of all elements in the output of the discriminator D at the spatial position 4. The unsupervised domain adaptation image semantic segmentation method based on task decomposition according to claim 3, characterized in that, An attribute discriminator D is added in the output layer out The attribute discriminator D is added in the output layer wherein, represents the discriminator D out the number of all elements in the output of the discriminator D at the spatial position.

5. The task deconstruction based unsupervised domain adaptation image semantic segmentation method according to claim 4, characterized in that, establishing the discriminator D shall and D out the discriminative loss of D and D respectively is 6. The unsupervised domain adaptation image semantic segmentation method based on task decomposition according to claim 1, characterized in that, The consistency loss is calculated by using a 2 norm, and a formula is as follows:

7. The task deconstruction based unsupervised domain adaptation image semantic segmentation method according to claim 1, characterized in that, The uncertainty value of the i th element in the U can be obtained by the following formula: wherein, represents p T the predicted value of the cth class at position i, the log function with base C of the total number of classes ensures that U i takes values between 0 and 1 ; U i reflects the uncertainty of the model for the predicted result , the closer to 0 the more certain the network, and vice versa. Under the guidance of the confidence map, the consistency loss is updated as: wherein, ⊙ represents element-wise multiplication; Thus, the total loss for training the entire segmentation model is: where λ1, λ2, and λ3 are used to balance the contribution of each loss term.

Images