Unsupervised domain adaptation semantic segmentation method, system, device and medium based on inter-domain consistency

Abstract

The unsupervised domain self-adaptive semantic segmentation method, system, device and medium based on inter-domain consistency use a multi-scale generative adversarial network based on GAN to translate source domain image data to target domain style while preserving its content; then input the inter-domain consistency training network based on the Transformer model for adaptive training, including sending the source domain image data after data enhancement to the student network for source domain semantic segmentation training and updating the teacher network weight using the EMA strategy; input the target domain unlabeled image data into the teacher network to generate pseudo labels to further guide the student network training; use a hybrid strategy to randomly combine the source domain image data and the target domain image data to guide the student network to learn domain invariant knowledge; finally, the student network is optimized according to the attention feature map consistency, and an unsupervised domain self-adaptive semantic segmentation model is output, which can complete the segmentation of the target domain image data without using the target domain label; the present application can train a model with excellent performance using less real image data set in the field of unmanned driving, greatly reducing the time and money cost of training, and improving the work efficiency.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of unmanned perception systems and transfer learning, and particularly relates to an unsupervised domain adaptation semantic segmentation method, system, device and medium based on inter-domain consistency. BACKGROUND

[0002] In the field of unmanned visual perception technology, existing models based on Transformer architecture have already outperformed models based on traditional CNN architecture in performance. Training the most advanced semantic segmentation model based on Transformer architecture for unmanned perception systems requires large-scale manually labeled image training data. However, in the real world, collecting and labeling training data is very expensive, requiring a lot of time and money. Therefore, training models using virtual synthetic image datasets has become a more promising solution, as it is easier to obtain labeled information than real image data.

[0003] However, when a model trained on a virtual synthetic image dataset directly predicts a real-world image dataset, it usually performs poorly, because there is a domain difference between the two image datasets, and the model is very sensitive to this. In order to reduce the performance gap, unsupervised domain adaptation (UDA) methods have been proposed. Unlabeled real-world images are used to adjust the model trained on synthetic images, so that the model is more suitable for the target domain.

[0004] Unsupervised domain adaptation technology is a new solution to the problems of expensive and time-consuming labeled image data in the field of unmanned perception. However, while training models using virtual synthetic image data brings convenience, this technology still presents several important challenges in the following aspects:

[0005] 1) There is a huge domain difference between virtual synthetic image datasets and real image datasets, and existing methods still cannot completely overcome this difference.

[0006] 2) Limited by the capacity of the model, existing methods perform poorly on small targets.

[0007] With the development of artificial intelligence and the increasing number of unsupervised domain adaptation techniques being proposed, many emerging deep learning-based algorithms are widely used in the field of unsupervised domain adaptation semantic segmentation. As a new technology in the field of unmanned driving, unsupervised domain adaptation technology can greatly reduce the demand for manually labeled image data for training unmanned algorithms, thereby greatly saving manpower and resources, while also improving the accuracy of perception models.

[0008] Most of the existing unsupervised domain adaptive semantic segmentation methods only focus on a single spatial adaptation of neural networks (such as input space, feature space and output space), and almost no method proposes a unified training framework to reduce the domain difference from multiple spaces to improve model performance.

[0009] The HRDA: Context-Aware High-Resolution Domain-Adaptive Semantic Segmentation of the European Conference on Computer Vision (ECCV) conference paper in 2022 adopts a multi-resolution training method, which combines high-resolution image cropping to retain fine segmentation details and low-resolution image cropping to capture the advantages of learning long-range context dependencies to improve segmentation accuracy, but this method does not consider the construction of intermediate domains and the construction of attention mechanism adaptation to improve the learning ability of the model, and has the shortcomings of poor small target segmentation accuracy and weak confusion class discrimination ability. SUMMARY

[0010] In order to overcome the defects of the prior art, the purpose of the present application is to provide an unsupervised domain adaptive semantic segmentation method, system, device and medium based on inter-domain consistency, which can train a semantic segmentation model on a labeled virtual synthetic image dataset (source domain image data) and achieve high accuracy prediction on real image dataset through auxiliary training of unlabeled real dataset (target domain image data); greatly reducing the time and money cost of training, with the advantages of fast, accurate and efficient.

[0011] In order to achieve the above purpose, the technical scheme adopted by the present application is:

[0012] An unsupervised domain adaptive semantic segmentation method based on inter-domain consistency, specifically comprising the following steps:

[0013] Step 1: input the labeled source domain image data into the GAN-based multi-scale image translation network for style transfer, thereby constructing the intermediate domain image data to reduce the distribution difference between the source domain and the target domain image data;

[0014] Step 2: input the intermediate domain image data constructed in step 1 into the inter-domain consistency training network based on the Transformer model, and output the semantic segmentation prediction result for the target domain real image dataset through the joint adjustment of the target domain image data and the intermediate domain image data.

[0015] The style refers to the visual characteristics of an image, including color scheme, texture, line form, lighting conditions, and detail complexity. Style transfer refers to the transfer of these characteristics from one image to another without changing the content of the image.

[0016] The specific method of step one is:

[0017] Input the labeled source domain image data into the GAN-based multi-scale image translation network to translate to an intermediate domain with the same content as the source domain image data but with a style consistent with that of the target domain image data:

[0018] 1.1: Use computer-generated simulated street view images as source domain image data, i.e., a virtual synthetic data set; use real images taken in real scenes as target domain, i.e., a real image data set;

[0019] 1.2: Assume that the GAN-based multi-scale image translation network has n scaling scales, with n being a human-selected value. Each scale has a CNN-based generator and a CNN-based discriminator. Each generator is responsible for translating image data at that scale. Each generator has two inputs: the first is the generated image data from the previous generator scaled up to the current scaling size, and the second is a scaled version of the source domain image data matching the current generator scale. The generator at the lowest scale only receives the second input. The output of each generator is translated image data at the current scale with source domain content and target domain style.

[0020] 1.3: Each discriminator has two inputs: the first is the output image data from the generator at the current scale, and the second is a scaled version of the target domain image data matching the current scale.

[0021] Starting from the generator and discriminator at the lowest scale, use the loss function to train the discriminator and generator sequentially. Finally, when reaching scale n, the final output of the generator is the intermediate domain image data with source domain image data content and target domain style.

[0022] The specific method of step two is:

[0023] Use a self-learning training method to simultaneously train two semantic segmentation networks based on the Transformer structure, which are identical in structure. One is a student network that normally accepts gradient descent training, and the other is a teacher network that is periodically updated based on the parameters of the student network.

[0024] 2.1: The student network first accepts the translated image data X generated in step one IAs input, and randomly crop it into m x m pixel patches, apply data augmentation methods including color jitter, Gaussian blur and ClassMix to the patch to obtain the intermediate domain image data after data augmentation; send the intermediate domain image data after data augmentation to the student network f θ The training is performed on the source domain semantic segmentation, and the loss function of the source domain semantic segmentation part is:

[0025]

[0026] wherein, is a weight parameter, equal to N is the total number of pixels, N c is the predicted number of pixels corresponding to the true value ground truth, which means that the pixel belongs to the predicted specific class c; is a one-hot vector, since the image translation module does not change the content and structure information of the image data, the ground truth of the translated image data remains unchanged as the source image data; when the prediction result is correct, i.e. then when the prediction result is wrong is the probability that the pixel belongs to class c, which is the prediction result of the student network, further represented as:

[0027] The teacher network and the student network share the same architecture, and the teacher network is not updated by gradient descent during the training process of the student network, but the EMA strategy is used to update the parameters of the teacher network:

[0028]

[0029] wherein, and θ are the parameters of the teacher network and the student network respectively; β is a hyperparameter for adjusting the update speed;

[0030] 2.2: Mix the intermediate domain image data generated in step one with the target domain image data to promote more stable training; select the complementary parts of the source domain image data and the target domain image data for mixing:

[0031] x Mix =M I x I +M T x T

[0032] wherein, M I is a mask, M T is the complementary part of the mask; based on the pseudo label generated by the teacher network, generate a mixed pseudo label with the corresponding part as a supervision signal:

[0033]

[0034] where y I is equal to the source domain label y S ;

[0035] The student network is trained using mixed image data and mixed pseudo labels, and the mixed cross-entropy loss is as follows:

[0036]

[0037] is the prediction result of the teacher network, and is further expressed as

[0038] 2.3: First, extract the attention feature maps corresponding to the intermediate domain image data and the target domain image data input into the student network in step two S1 from the decode head of the current student network, and extract the attention feature maps of the mixed image data from the decode head of the teacher network The attention features of the intermediate domain and the target domain image data are mixed by the mask M:

[0039]

[0040] where M is obtained by resizing the image mask M I ;

[0041] The Kullback-Leibler divergence is minimized to minimize and the difference between the two attention feature maps:

[0042]

[0043] The global loss function is composed of two pixel-level cross-entropy losses and attention adaptation losses, and the inter-domain consistency training network is trained under the constraint of the three losses:

[0044]

[0045] The inter-domain consistency training network finally outputs the student network as an adaptive semantic segmentation network to complete the semantic segmentation task on the target domain image data set.

[0046] An unsupervised domain adaptive semantic segmentation system based on inter-domain consistency, comprising:

[0047] A multi-scale image translation network module is used in step 1 to translate the labeled source domain image data, i.e. the virtual synthetic image data set, to an intermediate domain which has the same content as the source domain image but the style is consistent with the target domain, i.e. the real image data set;

[0048] An inter-domain consistency training network module is used in step 2 to simultaneously train two semantic segmentation networks based on the Transformer structure which are completely the same in structure, one of which is a student network normally trained by gradient descent, and the other is a teacher network periodically updated according to the parameters of the student network, and finally the student network is output as an adaptive semantic segmentation network to complete the semantic segmentation task on the target domain image data set.

[0049] An unsupervised domain adaptive semantic segmentation device based on inter-domain consistency comprises:

[0050] A memory for storing a computer program;

[0051] A processor for implementing the unsupervised domain adaptive semantic segmentation method based on inter-domain consistency as described in steps 1-2 when executing the computer program.

[0052] A computer readable storage medium stores a computer program, characterized in that the computer program is executed by a processor to implement the unsupervised domain adaptive semantic segmentation based on the inter-domain consistency method based on steps 1-2.

[0053] Compared with the prior art, the present application has the following advantages:

[0054] 1. The present application is a new breakthrough of the deep learning algorithm based on the Transformer architecture in the unsupervised domain adaptive semantic segmentation. The existing methods usually only focus on minimizing the inter-domain difference at a single level (such as pixel, feature and prediction), ignoring the mutual promotion of knowledge learning between different levels. The method proposed in the present application combines the knowledge learning at the pixel, feature and prediction levels, and proposes a complete self-training framework to output a domain adaptive semantic segmentation model with high accuracy.

[0055] 2. The image translation module is used to map the source domain image data to an intermediate domain with similar distribution as the target domain image data, to reduce the huge distribution difference between the source domain and the target domain. At the same time, two kinds of pixel-level cross-entropy loss and attention adaptation loss are combined, and the semantic segmentation model learns the domain-invariant knowledge shared by the source domain and the target domain, and can accurately and efficiently apply the model trained on the virtual synthetic image data set to the real world data set.

[0056] 3. Through quantitative verification carried out in the GTA to Cityscapes experimental setup, the application improves +1.3mIoU compared with the existing optimal method in the Cityscapes real street view image dataset without using real image dataset label data, reaches the most advanced performance, and has excellent segmentation effect for small targets and easily confused classes. As a new method in the field of autonomous driving perception, the application can save a lot of manual labeling cost, reduce the training difficulty of the autonomous driving semantic segmentation model, and improve the accuracy in practical application.

[0057] In summary, the application can realize high-accuracy prediction on real image datasets through the auxiliary training of unlabeled real datasets (target domain image data) for semantic segmentation models trained on labeled virtual synthetic image datasets (source domain image data). This greatly reduces the time and money cost of training, and has the advantages of being fast, accurate and efficient. BRIEF DESCRIPTION OF DRAWINGS

[0058] Figure 1 The semantic segmentation network adaptive training block diagram proposed by the application.

[0059] Figure 2 The visualization schematic diagram of step one of the application.

[0060] Figure 3 The visualization schematic diagram of step one of the application.

[0061] Figure 4 The visualization schematic diagram of step two 2.1 of the application.

[0062] Figure 5 The visualization schematic diagram of step two 2.2 of the application.

[0063] Figure 6 The visualization schematic diagram of step two 2.3 of the application. DETAILED DESCRIPTION

[0064] The application will be further described in detail below with reference to the accompanying drawings.

[0065] Referring to Figure 1 An unsupervised domain adaptive semantic segmentation method based on inter-domain consistency, specifically comprising the following steps:

[0066] Step one: input the labeled source domain image data into the multi-scale image translation network based on GAN for style transfer, thereby constructing the intermediate domain image data to reduce the distribution difference between the source domain and the target domain image data;

[0067] The style refers to the visual characteristics of an image, including color scheme, texture, line form, lighting conditions, and detail complexity, and the style transfer refers to the transfer of the above characteristics from one image to another without changing the image content.

[0068] Referring to Figure 2 , the specific method of step one is:

[0069] 1.1: Use computer-generated simulated street view images as source domain image data, i.e., a virtual synthetic data set; use real image data taken from real scenes as target domain, i.e., a real image data set;

[0070] 1.2: Referring to Figure 3 , assume that the GAN-based multi-scale image translation network has n scaling scales, and n is artificially selected. Each scale has a CNN-based generator and a CNN-based discriminator. Each generator is responsible for translating image data at that scale. Each generator at each scale has two inputs: the first is the generated image data from the generator at the previous scale scaled up to the current scaling size, and the second is a scaled version of the source domain image data matching the current generator scale. The generator at the lowest scale only receives the second input. The output of each scale generator is a translated image data at the current scale with source domain content and target domain style.

[0071] 1.3: Each scale discriminator has two inputs: the first is the output image data of the generator at the current scale, and the second is a scaled version of the target domain image data matching the current scale.

[0072] Starting from the generator and discriminator at the lowest scale, use the loss function and L G =-E(logD(G(z))) to train the discriminator and generator in turn, where D(x) refers to the output of the discriminator at that scale for real image data x, and G(z) is the output of the generator at that scale for two inputs; finally, when reaching scale n, the final output of the generator is the intermediate domain image data with source domain image data content and target domain style.

[0073] Step two: input the intermediate domain image data constructed in step one into the domain consistency training network based on the Transformer model, and output the semantic segmentation prediction results for the target domain real image data set through the joint adjustment of the target domain image data and the intermediate domain image data.

[0074] The specific method is:

[0075] The training method of self-learning is adopted to train two semantic segmentation networks based on the Transformer structure which are completely the same, one of which is a student network normally trained by gradient descent, and the other is a teacher network periodically updated according to the parameters of the student network;

[0076] Referring to Figure 4 , 2.1: the student network first accepts the translated image data X generated in step one I as input, and randomly crops it into m x m pixel patches, applies color jitter, Gaussian blur and ClassMix data augmentation methods to enhance the patch, and obtains the intermediate domain image data after data augmentation; the intermediate domain image data after data augmentation is sent to the student network f θ The training is performed for source domain semantic segmentation, and the loss function of the source domain semantic segmentation part is:

[0077]

[0078] wherein, is a weight parameter, equal to N is the total number of pixels, N c is the predicted number of pixels corresponding to the true value ground truth, which means that the pixel belongs to the predicted specific class c; is a one-hot vector, since the image translation module does not change the content and structure information of the image data, the ground truth of the translated image data remains unchanged with the source image data; when the prediction result is correct, i.e. then when the prediction result is wrong This is because in the semantic segmentation cross-entropy loss, we only care about the prediction probability of the predicted correct class, and by training, we increase the prediction probability of the model for the predicted correct class, because the sum of the prediction probabilities for all classes is 1, the prediction probability of the remaining predicted wrong classes will be indirectly reduced, and the accuracy of the model is improved; is the probability that the pixel belongs to class c, which is the prediction result of the student network, and is further expressed as:

[0079] The teacher network and the student network share the same architecture, and the teacher network is not updated by gradient descent weight during the training process of the student network, but the EMA strategy is used to update the parameters of the teacher network:

[0080]

[0081] wherein, and 0 are the parameters of the teacher network and student network, respectively; 0 is a hyper-parameter for adjusting the update speed; through the EMA update strategy, the prediction of the teacher network changes steadily, which ensures the smooth progress of the training process;

[0082] Referring to Figure 5 , 2.2: mixing the intermediate domain image data generated in step one with the target domain image data to promote more stable training; selecting the complementary parts of the source domain image data and the target domain image data for mixing:

[0083] x Mix =M I x I +M T x T

[0084] wherein M I is a mask, M T is the complementary part of the mask; based on the pseudo-label generated by the teacher network, a mixed pseudo-label with the corresponding part is generated as a supervision signal:

[0085]

[0086] wherein y I is equal to the source domain label y S ;

[0087] The student network is trained using the mixed image data and the mixed pseudo-label, and the mixed cross-entropy loss is as follows:

[0088]

[0089] Different from S1, here is the prediction result of the teacher network, which can be further expressed as

[0090] Referring to Figure 6 , 2.3: first, the attention feature maps corresponding to the intermediate domain image data and the target domain image data input into the student network in step two S1 are extracted from the decode head of the current student network, and the attention feature maps of the mixed image data are extracted from the decode head of the teacher network The attention features of the intermediate domain and the target domain image data are mixed through the mask M:

[0091]

[0092] wherein M is obtained by resizing the image mask M I ;

[0093] The Kullback-Leibler divergence is minimized to minimize and Difference between two attention feature maps:

[0094]

[0095] The global loss function is composed of two pixel-level cross-entropy losses and an attention adaptation loss, and the inter-domain consistency training network is trained under the constraint of the three losses:

[0096]

[0097] The final inter-domain consistency training network outputs the student network as an adaptive semantic segmentation network to complete the semantic segmentation task on the target domain image dataset.

[0098] An unsupervised domain adaptive semantic segmentation system based on inter-domain consistency, comprising:

[0099] A multi-scale image translation network module, in step 1, is used to translate the labeled source domain image data (virtual synthetic image dataset) to an intermediate domain with the same content as the source domain image data but consistent with the target domain (real image dataset) in style.

[0100] An inter-domain consistency training network module, in step 2, is used to simultaneously train two structure-identical semantic segmentation networks based on the Transformer structure, one of which is a student network trained normally with gradient descent, and the other is a teacher network periodically updated according to the student network parameters, and finally outputs the student network as an adaptive semantic segmentation network to complete the semantic segmentation task on the target domain image dataset.

[0101] An unsupervised domain adaptive semantic segmentation device based on inter-domain consistency, comprising:

[0102] A memory for storing a computer program;

[0103] A processor for implementing the unsupervised domain adaptive semantic segmentation method based on inter-domain consistency as described in steps 1-2 when executing the computer program.

[0104] A computer-readable storage medium storing a computer program, characterized in that the computer program, when executed by a processor, can implement unsupervised domain adaptive semantic segmentation based on the unsupervised domain adaptive semantic segmentation method based on inter-domain consistency as described in steps 1-2.

[0105] Experimental verification

[0106] We test the proposed method on GTA→Cityscapes experimental setup. GTA V is a synthetic image dataset with pixel-level annotation generated by game engine, containing 24966 images with resolution of 1914x1052. Cityscapes is a real-world street scene image dataset, containing 2975 training set images and 500 validation set images for evaluation.

[0107] In GTA→Cityscapes, the proposed method has a significant improvement compared to existing methods. When the method is applied to the HRDA network structure, the mIoU reaches 75.1, which is 1.3 mIoU higher than HRDA. When the method is applied to the MIC network structure, it improves by +1.3 mIoU, reaching the current state-of-the-art performance of 77.2 mIoU, proving the effectiveness of the method. Moreover, the method improves the accuracy of the traffic light and bicycle classes by +3.2 IoU and +4.0 IoU respectively, proving that the method helps to improve the discriminability of easily confused classes, especially for small targets.

Claims

1. An unsupervised domain-adaptive semantic segmentation method based on inter-domain consistency, characterized in that, Specifically, the following steps are included: Step 1: Input the labeled source domain image data into a GAN-based multi-scale image translation network, translate it to an intermediate domain with the same content as the source domain image data but with the same style as the target domain image data, perform style transfer, and construct intermediate domain image data in this way to reduce the distribution difference between the source domain and target domain image data. Step 2: Input the intermediate domain image data constructed in Step 1 into the inter-domain consistency training network based on the Transformer model. Through joint adjustment of the target domain image data and the intermediate domain image data, output the semantic segmentation prediction result for the target domain real image dataset; specifically: A self-learning training method is adopted to train two semantic segmentation networks with identical structures based on the Transformer structure at the same time. One is a student network that is trained normally by gradient descent, and the other is a teacher network that is periodically updated according to the parameters of the student network. 2.1: The student network first receives the translated image data X generated in step one. I As input, the data is randomly cropped into m×m pixel patches. These patches are then augmented using data augmentation methods including color dithering, Gaussian blur, and ClassMix, resulting in augmented intermediate domain image data. This augmented intermediate domain image data is then sent to the student network f. θ Training performs source domain semantic segmentation, and the loss function for the source domain semantic segmentation part is: in, It is a weight parameter, equal to N is the total number of pixels, N c It is the number of predicted pixels corresponding to the ground truth, meaning that the pixel belongs to the predicted specific category c; It is a one-hot vector. Since the image translation module does not change the content and structural information of the image data, the ground truth of the translated image data remains unchanged from the source image data; when the prediction result is correct... but If the prediction result is incorrect, then Let be the probability that a pixel belongs to class c, which is the prediction result of the student network, and further expressed as: The teacher network shares the same architecture as the student network, and instead of updating the teacher network's weights using gradient descent during the student network's training, the EMA strategy is used to update the teacher network's parameters. in, θ and θ are the parameters of the teacher network and student network, respectively; β is a hyperparameter used to adjust the update speed. 2.2: The intermediate domain image data generated in step one is mixed with the target domain image data to promote more stable training; complementary parts of the source domain image data and target domain image data are selected for mixing: X Mix =M I X I +M T X T Among them, M I It's a mask, M T It is the complementary part of the mask; based on the pseudo-labels generated by the teacher network, a hybrid pseudo-label with the corresponding part is generated as a supervision signal: Among them, Y I Equal to source domain label y S ; The student network is trained using mixed image data and mixed pseudo-labels, with the mixed cross-entropy loss as follows: The prediction results for the teacher network are further expressed as 2.3: First, extract the attention feature map corresponding to the intermediate domain image data input into the student network in step two S1 from the decode head of the current student network. Attention feature map corresponding to the target domain image data Simultaneously, attention feature maps of the mixed image data are extracted from the decode head of the teacher network. Attention features are blended between intermediate and target domain image data using a mask M: Where M is the image mask M I Obtained by adjusting the size; Minimize Kullback-Leibler divergence to minimize and Differences between the two attention feature maps: The global loss function consists of two pixel-level cross-entropy losses and an attention adaptation loss. The inter-domain consistency training network is trained under the constraints of these three losses: Finally, the inter-domain consistency training network outputs a student network as an adaptive semantic segmentation network to complete the semantic segmentation task on the target domain image dataset.

2. The unsupervised domain-adaptive semantic segmentation method based on inter-domain consistency according to claim 1, characterized in that, In the style transfer, style refers to the visual characteristics of an image, including color scheme, texture, line form, lighting conditions, and detail complexity. Style transfer refers to the transfer of these characteristics from one image to another without changing the image content.

3. The unsupervised domain-adaptive semantic segmentation method based on inter-domain consistency according to claim 1, characterized in that, The specific method for step one is as follows: Labeled source domain image data is input into a GAN-based multi-scale image translation network, which translates the data into an intermediate domain with the same content as the source domain image data but a style consistent with the target domain image. 1.1: Simulated street scene images artificially generated by computer engine are used as the source domain image data, i.e., virtual synthetic image dataset; real image data collected from real scene roads are used as the target domain, i.e., real image dataset. 1.2: Suppose a GAN-based multi-scale image translation network has n scaling scales, where the value of n is chosen manually. Each scale has a CNN-based generator and a CNN-based discriminator. Each generator is responsible for translating image data at that scale. Each scale's generator has two inputs: the first is the generated image data from the previous scale's generator, scaled up to the current scaling ratio; the second is a scaled version of the source domain image data matching the current generator's scale. The generator at the lowest scale only receives the second input. The output of each scale's generator is the translated image data at the current scale, which has source domain content and target domain style. 1.3: The discriminator at each scale has two inputs: the first is the output image data of the generator at the current scale, and the second is a scaled version of the target domain image data that matches the current scale; Starting with the lowest-level generator and discriminator, using a loss function The discriminator and generator are trained sequentially. When the scale n is reached, the final output of the generator is the intermediate domain image data with the source domain image data content and the target domain style.

4. An unsupervised domain-adaptive semantic segmentation system based on inter-domain consistency, characterized in that, Based on the method described in claims 1 to 3, it includes: The multi-scale image translation network module, in step 1, is used to translate the labeled source domain image data, i.e., the virtual synthetic image dataset, into an intermediate domain with the same content as the source domain image data, but with the same style as the target domain, i.e., the real image dataset. In step 2, the inter-domain consistency training network module is used to simultaneously train two semantic segmentation networks with identical structures based on the Transformer architecture. One is a student network that undergoes normal gradient descent training, and the other is a teacher network that is periodically updated according to the parameters of the student network. Finally, the student network is output as an adaptive semantic segmentation network to complete the semantic segmentation task on the target domain image dataset.

5. An unsupervised domain-adaptive semantic segmentation device based on inter-domain consistency, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the unsupervised domain-adaptive semantic segmentation method based on inter-domain consistency as described in claims 1 to 3 when executing the computer program.

6. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it can perform unsupervised adaptive semantic segmentation based on the unsupervised domain adaptive semantic segmentation method based on inter-domain consistency as described in claims 1 to 3.

Images