Domain adaptation semantic segmentation method fusing adversarial training and self-training

Abstract

The invention provides a domain adaptive semantic segmentation method fusing adversarial training and self-training, and relates to the field of image processing, and the method comprises the steps: obtaining source domain data and target domain data, and carrying out the preprocessing; constructing a semantic segmentation model; pre-training the semantic segmentation model through the source domain data; carrying out first-stage adversarial training on the basis of a pre-trained semantic segmentation model through the source domain data and the target domain data; generating a pseudo tag of the target domain data through the semantic segmentation model after adversarial training; performing second-stage adversarial training on the basis of the first-stage adversarial training through the source domain data and the target domain data containing the pseudo labels; obtaining a target data set to be subjected to semantic segmentation; and inputting the target data set into the semantic segmentation model after adversarial training in the second stage to complete semantic segmentation of the target data set. And two random classifiers are used for replacing a plurality of classifiers, so that higher precision can be realized with fewer parameters.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to a domain adaptation semantic segmentation method integrating adversarial training and self-training. Background Art

[0002] Deep learning can learn from massive amounts of data and automatically mine features in the data. The most representative model in deep learning is the convolutional neural network, which extracts features from the data through multiple convolution operations and pooling operations, maps low-level features to high-level features, and classifies high-level features to complete image processing. Some scholars have achieved pixel-level classification by replacing the fully connected layer of the convolutional network with a convolutional layer, thereby applying deep learning to semantic segmentation. Currently, semantic segmentation mainly includes convolutional neural networks based on convolution and Transformer network structures based on attention mechanisms.

[0003] Although the two models can achieve good results in semantic segmentation, model training often requires a large amount of labeled data, and the labeling of image data usually consumes a lot of manpower and material resources. In addition, due to the regional characteristics and acquisition instrument differences between different data sets, the effect of model training on one data set may not be directly transferred to another data set. In order to solve this problem, scholars at home and abroad have proposed various domain adaptation methods. The existing domain adaptation methods mainly include adversarial training and self-training. Adversarial training mainly tries to make the feature distribution of the source domain and the target domain closer by minimizing the adversarial loss. The self-training method generates pseudo labels for the target data based on the source domain data and trains on the basis of the pseudo labels.

[0004] The Chinese patent "Fundus image segmentation method, system and application based on unsupervised domain adaptation and consistent self-supervised learning, application number CN202410632826.3" discloses a domain adaptation method for semantic segmentation, which improves the accuracy of semantic segmentation through unsupervised domain adaptation and self-supervised learning. In this method, Fourier transform is used to align image features, and the consistency of image prediction is evaluated in combination with a discriminator network. Pseudo-labels are generated through self-supervision and trained in combination with category boundary constraints, thereby improving segmentation accuracy.

[0005] The Chinese patent "Hyperspectral image classification method based on pseudo-label assistance, application number 202311004129.5" discloses a hyperspectral image classification method based on pseudo-label assistance, which aims to improve the accuracy of cross-domain classification. The method uses a deep neural network to extract features, predicts through two classifiers, and minimizes the maximum mean difference loss function to update the model. After using the model to obtain the target domain pseudo-label, continue to optimize. This method improves the accuracy of hyperspectral image classification.

[0006] The Chinese patent "A method and system for measuring the difference between multiple classifiers based on confusion matrix, application number 201911077222.2" discloses a method for measuring the difference based on confusion matrix. By evaluating the confusion matrix of each sub-classifier, the correlation between sub-classifiers is calculated, and the difference between classifiers is evaluated accordingly. This method can effectively improve the correlation between the fusion classification performance and difference measurement values ​​of the classifier set.

[0007] Although current domain adaptation methods can improve the prediction accuracy of semantic segmentation models on target domain data, there are still several limitations in existing methods: 1. Traditional domain adaptation methods are mostly used in image classification tasks, while semantic segmentation is a pixel-level task. The information contained in the data is richer and more complex, and the requirements for labels are also higher. When the target domain data is unlabeled, how to effectively use the information in the labeled source domain data and target domain data for training to achieve high-precision semantic segmentation of the target domain data by the model is a problem that needs to be solved urgently.

[0008] 2. Adversarial training in domain adaptation methods usually focuses on distinguishing the features of the source domain and the target domain, while ignoring the relationship between the target sample and the classification decision boundary, so that the generator only tries to make the feature distribution similar, and it is easy to generate classification fuzzy features near the boundary. In the early stages of training, the pseudo labels generated by the self-training method may not be of high quality, and incorrect pseudo labels may have a negative impact on the model.

[0009] 3. Some scholars have proposed that the use of multiple classifiers can further improve the accuracy of the model, but the more classifiers there are, the more parameters the model stores. How to use this method to improve accuracy while minimizing the number of parameters is also a question that needs to be considered. Summary of the invention

[0010] The purpose of the present invention is to provide a domain adaptation semantic segmentation method that integrates adversarial training and self-training in order to solve the problem of poor accuracy of existing domain adaptation segmentation methods.

[0011] The above-mentioned purpose of the present application is achieved through the following technical solutions: S1: Obtain source domain data and target domain data and perform preprocessing; S2: Build a semantic segmentation model; S3: Pre-train the semantic segmentation model using source domain data; S4: The first stage of adversarial training is performed on the basis of the pre-trained semantic segmentation model using source domain data and target domain data. S5: Generate pseudo labels for target domain data through the adversarially trained semantic segmentation model; S6: Perform the second stage of adversarial training based on the first stage of adversarial training using source domain data and target domain data with pseudo labels; S7: Obtain a target data set to be semantically segmented; input the target data set into the semantic segmentation model after the adversarial training in the second stage to complete the semantic segmentation of the target data set.

[0012] Optionally, step S1 includes: S11: Acquire a data set with labels as source domain data, and read a first image picture and a label picture from the source domain data; S12: Crop the first image and the label image into a size of The image is normalized. S13: Obtain the data set to be predicted as the target domain data, read the second image picture from the target domain data; crop the second image picture into The image is normalized and the Normalize operation is performed.

[0013] Optionally, step S2 includes: Build a semantic segmentation model; Use the downsampled part of the semantic segmentation model as the feature extractor G; Use the upsampled part of the semantic segmentation model as a classifier ; Classifier-based Introduce Gaussian distribution to randomize the convolution weights to form a random classifier ; In the initialization of the semantic segmentation model, instantiate a feature extractor G and a classifier and two random classifiers , .

[0014] Optionally, step S3 includes: S31: The preprocessed source domain data Input feature extractor G, feature extractor G outputs feature ; S32: Source domain features Input random classifiers respectively , , , according to the predicted probabilities output by the three classifiers , , Calculate segmentation loss ; S33: By minimizing the segmentation loss To optimize feature extraction G and , , :

[0015] in is the size of the source domain image, Represents the number of categories of source domain data, is the true label value corresponding to the source domain data; S34: Repeat steps S31 to S33 until the set maximum number of iterations is reached.

[0016] Optionally, step S4 includes: S41: Input the preprocessed source domain data and target domain data into the feature extractor G to obtain the output source domain features respectively , target domain features ; The source domain features Input classifier and a random classifier , The predicted probability , , , calculate the segmentation loss , and optimize G, , , ; S42: Target domain features Input Random Classifier , The predicted probability , ,calculate , The difference loss :

[0017]

[0018]

[0019] in Represents the number of categories of target domain data, and the difference loss is given by and It consists of two parts. represents the joint classification deterministic loss function, represents the local classification deterministic loss function; By maximizing the difference loss To optimize the feature extractor G; S43: By minimizing the difference loss Optimizing random classifiers , ,as follows: The target domain features Input Random Classifier , Get the new predicted probability , and the new difference loss , continue to calculate and minimize the difference loss To continuously optimize the random classifier , ; S44: Take the predicted probability obtained in step S41 , , calculate the predicted probability and predicted probability The average value is taken as the centralized prediction result of the random classifier on the source domain data ; The dimension is The probability value and dimensions are The probability value Concatenate along the last dimension to get a dimension of Tensor ; right The last dimension of The probability and the following The probability and of the bits are combined to obtain a shape of Tensor ; The tensor As source domain features Field label ; The last dimension has Bit; S45: Take the predicted probability calculated in step S43 , The average value is used as the centralized prediction result of the random classifier on the target domain data ; The target domain features Input classifier Get the predicted probability ; according to , get the target domain features Field label ; S46: Use 1 to represent data from the source domain and 0 to represent data from the target domain to create a real domain label ,calculate , and The sum of the cross-loss entropy is used as the domain loss :

[0020] in, Represents source domain data; Represents the target domain data; S47: By minimizing the domain loss Optimizing the classifier , Random Classifier , Random Classifier , and feature extractor G.

[0021] Optionally, step S5 includes: S51: traverse each batch of target domain data in the target domain data loader, each batch contains only one image; S52: Use the adversarially trained semantic segmentation model to extract and predict features for each image; The random classifier of the semantic segmentation model predicts the features of each image k times and takes the average as the final prediction result; The probability map is calculated by upsampling and normalizing the final prediction result; By performing the maximum value operation on each pixel of the probability map, the confidence value of each pixel belonging to different categories is obtained; Store each confidence value in the category's confidence dictionary; S52: sorting the confidence list of each category in the confidence dictionary in descending order, and for each category, taking the value of the eightieth percentile position from the sorted list as the category threshold of the category; S53: traverse the target domain data loader, and load the corresponding probability map for each batch of target domain data; The final prediction result of each image is divided by the category threshold to calculate the weighted probability and determine the category label of each pixel in the image; pixels with a weighted probability less than 1 are marked as 255, and 255 is set as an ignorable label and does not participate in the calculation of the loss function; S54: The category label of each pixel in the image is saved as a pseudo label in the form of an image in a specified path.

[0022] Optionally, step S6 includes: S61: re-read the target domain data containing the pseudo-labels, so that the target domain data loader contains the target domain data and the corresponding pseudo-labels; S62: Input the source domain data and the target domain data with pseudo labels into the feature extractor G to obtain the output source domain features respectively. , target domain features ; The source domain features Input classifier and a random classifier , The predicted probability , , , calculate the source domain segmentation loss , optimize the feature extractor G and classifier ; S63: Calculate the difference loss in the same way as step S42 , by maximizing the difference loss To optimize the feature extractor; S64: Same as step S43 by minimizing the difference loss For random classifier , Perform k optimizations; S65: Same as step S44 and step S45, calculate the integrated probability of the prediction results of the random classifier for the source domain data and the target domain data , , calculate the corresponding domain label prediction and domain loss ; Use the cross entropy loss function to calculate the target domain prediction results Segmentation loss with pseudo labels :

[0023] in Represents the target domain data, Represents the number of categories of target domain data, is the pseudo label corresponding to the target domain data; By minimizing the target domain segmentation loss and minimize the domain loss , for a random classifier , Optimize by minimizing the target domain segmentation loss and maximize the domain loss , optimize the feature extractor G.

[0024] An electronic device comprises a processor, a memory, a user interface and a network interface, wherein the memory is used to store instructions, the user interface and the network interface are used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device performs a domain adaptation semantic segmentation method that integrates adversarial training and self-training.

[0025] A computer-readable storage medium stores instructions. When the instructions are executed, a domain adaptation semantic segmentation method integrating adversarial training and self-training is performed.

[0026] The beneficial effects of the technical solution provided by this application are: 1. In the training process, adversarial training of classifier differences and adversarial training of domain label differences are integrated, so that the semantic segmentation model can better distinguish source domain data from target domain data, and correctly classify the data in the target domain. In addition, two random classifiers are used instead of multiple classifiers to achieve higher accuracy with fewer parameters.

[0027] 2. This method combines adversarial training of classifier differences with adversarial training of domain label differences and self-training. First, the classifier and feature extractor are optimized separately by minimizing the classifier differences, which improves the classifier's ability to distinguish and enables the feature extractor to extract features far away from the classification boundary. Subsequently, the integrated results of the two classifiers are processed with the prediction results of another classifier to obtain the domain label, and then the feature extractor and classifier are optimized by minimizing the domain loss. After the model has become relatively stable, a high-confidence pseudo-label is selected to add a self-training process. The two-stage adversarial training and self-training improve the accuracy of target domain data extraction from both the feature and domain levels. Two random classifiers are used in this method. Although only the parameters of these two classifiers need to be stored, multiple prediction results can be generated through multiple predictions to replace the predictions of multiple classifiers, thereby improving the prediction accuracy. At the same time, the classification certainty maximization method is introduced when calculating the differences between classifiers. On the basis of obtaining more possible prediction results, the certainty of the classifier in data classification is guaranteed. 3. A new way is used to calculate the difference loss between classifiers, which takes into account the joint certainty between classifiers and the individual certainty of each classifier, further improving the accuracy of the model.

[0028] 4. The design of the loss function also specifically considers the characteristic that each pixel of the image data in semantic segmentation has a corresponding probability value. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The present application will be further described below with reference to the accompanying drawings and embodiments, in which: Figure 1is a step diagram in an embodiment of the present application; Figure 2 It is the pre-training graph in the embodiment of the present application; Figure 3 is the adversarial training graph of the first stage in the embodiment of the present application; Figure 4 is the adversarial training graph of the second stage in the embodiment of the present application; Figure 5 It is a schematic diagram of the structure of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0030] In order to have a clearer understanding of the technical features, purposes and effects of the present application, the specific implementation methods of the present application are now described in detail with reference to the accompanying drawings.

[0031] The embodiments of the present application provide a domain adaptation semantic segmentation method that integrates adversarial training and self-training.

[0032] Please refer to Figure 1 , Figure 1 This is a step diagram of a domain adaptation semantic segmentation method integrating adversarial training and self-training in an embodiment of the present application, including: S1: Obtain source domain data and target domain data and perform preprocessing; Step S1 includes: S11: Acquire a data set with labels as source domain data, and read a first image picture and a label picture from the source domain data; S12: Crop the first image and the label image into a size of The image is normalized. S13: Obtain the data set to be predicted as the target domain data, read the second image picture from the target domain data; crop the second image picture into The image is normalized and the Normalize operation is performed.

[0033] S2: Build a semantic segmentation model; Step S2 includes: Build a semantic segmentation model; Use the downsampled part of the semantic segmentation model as the feature extractor G; Use the upsampled part of the semantic segmentation model as a classifier ; Classifier-based Introduce Gaussian distribution to randomize the convolution weights to form a random classifier ; Instantiate a feature extractor G and a classifier and two random classifiers , .

[0034] S3: Pre-train the semantic segmentation model using source domain data; Step S3 includes: S31: The preprocessed source domain data Input feature extractor G, feature extractor G outputs feature ; S32: Source domain features Input random classifiers respectively , , , according to the predicted probabilities output by the three classifiers , , Calculate segmentation loss ; S33: By minimizing the segmentation loss To optimize feature extraction G and , , :

[0035] in is the size of the source domain image, Represents the number of categories of source domain data, is the true label value corresponding to the source domain data; S34: Repeat steps S31 to S33 until the set maximum number of iterations is reached.

[0036] Specifically, Figure 2 As shown, the model is pre-trained using source domain data.

[0037] S4: The first stage of adversarial training is performed on the basis of the pre-trained semantic segmentation model using source domain data and target domain data. Step S4 includes: S41: Input the preprocessed source domain data and target domain data into the feature extractor G to obtain the output source domain features respectively , target domain features ; The source domain features Input classifier and a random classifier , The predicted probability , , , calculate the segmentation loss , and optimize G, , , ; S42: Target domain features Input Random Classifier , The predicted probability , ,calculate , The difference loss :

[0038]

[0039]

[0040] in Represents the number of categories of target domain data, and the difference loss is given by and It consists of two parts. represents the joint classification deterministic loss function, represents the local classification deterministic loss function; Specifically, by comprehensively considering the joint certainty between classifiers and the individual certainty of each classifier, the classifier can focus on the certainty of the classification results while paying attention to category differences.

[0041] By maximizing the difference loss To optimize the feature extractor G; S43: By minimizing the difference loss Optimizing random classifiers , ,as follows: The target domain features Input Random Classifier , Get the new predicted probability , and the new difference loss , continue to calculate and minimize the difference loss To continuously optimize the random classifier , ; Specifically, step S43 is repeated k times, and the feature extractor is enabled to better generate target features far away from the decision boundary by maximizing the minimization of the random classifier differences.

[0042] S44: Take the predicted probability obtained in step S41 , , calculate the predicted probability and predicted probability The average value is taken as the centralized prediction result of the random classifier on the source domain data ; The dimension is The probability value and dimensions are The probability value Concatenate along the last dimension to get a dimension of Tensor ; right The last dimension of The probability and the following The probability and of the bits are combined to obtain a shape of Tensor ; The tensor As source domain features Field label ; The last dimension has Bit; S45: Take the predicted probability calculated in step S43 , The average value is used as the centralized prediction result of the random classifier on the target domain data ; The target domain features Input classifier Get the predicted probability ; according to , get the target domain features Field label ; S46: Use 1 to represent data from the source domain and 0 to represent data from the target domain to create a real domain label ,calculate , and The sum of the cross-loss entropy is used as the domain loss :

[0043] in, Represents source domain data; Represents the target domain data; S47: By minimizing the domain loss Optimizing the classifier , Random Classifier , Random Classifier , and feature extractor G.

[0044] Specifically, by minimizing the domain label loss, the feature extractor G can further learn the invariant features of the data in such adversarial training. Figure 3 As shown in Figure 1, the first phase of adversarial training is performed on the basis of the pre-trained model using labeled source domain data and unlabeled target domain data. S5: Generate pseudo labels for target domain data through the adversarially trained semantic segmentation model; Step S5 includes: S51: traverse each batch of target domain data in the target domain data loader, each batch contains only one image; S52: Use the adversarially trained semantic segmentation model to extract and predict features for each image; The random classifier of the semantic segmentation model predicts the features of each image k times and takes the average as the final prediction result; The probability map is calculated by upsampling and normalizing the final prediction result; By performing the maximum value operation on each pixel of the probability map, the confidence value of each pixel belonging to different categories is obtained; Store each confidence value in the category's confidence dictionary; S52: sorting the confidence list of each category in the confidence dictionary in descending order, and for each category, taking the value of the eightieth percentile position from the sorted list as the category threshold of the category; S53: traverse the target domain data loader, and load the corresponding probability map for each batch of target domain data; The final prediction result of each image is divided by the category threshold to calculate the weighted probability and determine the category label of each pixel in the image; pixels with a weighted probability less than 1 are marked as 255, and 255 is set as an ignorable label and does not participate in the calculation of the loss function; S54: The category label of each pixel in the image is saved as a pseudo label in the form of an image in a specified path.

[0045] S6: Perform the second stage of adversarial training based on the first stage of adversarial training using source domain data and target domain data with pseudo labels; Step S6 includes: S61: re-read the target domain data containing the pseudo-labels, so that the target domain data loader contains the target domain data and the corresponding pseudo-labels; S62: Input the source domain data and the target domain data with pseudo labels into the feature extractor G to obtain the output source domain features respectively. , target domain features ; The source domain features Input classifier and a random classifier , The predicted probability , , , calculate the source domain segmentation loss , optimize the feature extractor G and classifier ; S63: Calculate the difference loss in the same way as step S42 , by maximizing the difference loss To optimize the feature extractor; S64: Same as step S43 by minimizing the difference loss For random classifier , Perform k optimizations; S65: Same as step S44 and step S45, calculate the integrated probability of the prediction results of the random classifier for the source domain data and the target domain data , , calculate the corresponding domain label prediction and domain loss ; Use the cross entropy loss function to calculate the target domain prediction results Segmentation loss with pseudo labels :

[0046] in Represents the target domain data, Represents the number of categories of target domain data, is the pseudo label corresponding to the target domain data; By minimizing the target domain segmentation loss and minimize the domain loss , for a random classifier , Optimize by minimizing the target domain segmentation loss and maximize the domain loss , optimize the feature extractor G.

[0047] Specifically, Figure 4 As shown, the second stage of adversarial training is performed on the basis of the first stage of adversarial training using source domain data with labels and target domain data with pseudo labels.

[0048] S7: Obtain a target data set to be semantically segmented; input the target data set into the semantic segmentation model after the adversarial training in the second stage to complete the semantic segmentation of the target data set.

[0049] The present application also discloses an electronic device. Figure 5 , Figure 5 The electronic device 500 may include: at least one processor 501 , at least one network interface 504 , a user interface 503 , a memory 505 , and at least one communication bus 502 .

[0050] The communication bus 502 is used to realize the connection and communication between these components.

[0051] The user interface 503 may include a display screen, and the optional user interface 503 may also include a standard wired interface or a wireless interface.

[0052] The network interface 504 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0053] The present application also discloses a computer-readable storage medium storing a plurality of instructions, which are suitable for loading by a processor to execute the above-mentioned domain adaptation semantic segmentation method integrating adversarial training and self-training.

[0054] The above are only exemplary embodiments of the present disclosure and cannot be used to limit the scope of the present disclosure. That is, any equivalent changes and modifications made according to the teachings of the present disclosure are still within the scope of the present disclosure. After considering the disclosure of the specification and the truth of practice, those skilled in the art will easily think of other embodiments of the present disclosure.

[0055] This application is intended to cover any variation, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary technical means in the art not described in the present disclosure. The description and examples are to be regarded as exemplary only, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. A domain adaptation semantic segmentation method integrating adversarial training and self-training, characterized in that: The method comprises the following steps: S1: Obtain source domain data and target domain data and perform preprocessing; S2: Build a semantic segmentation model; S3: Pre-train the semantic segmentation model using source domain data; S4: The first stage of adversarial training is performed based on the pre-trained semantic segmentation model using source domain data and target domain data. S5: Generate pseudo labels for target domain data through the adversarially trained semantic segmentation model; S6: Perform the second stage of adversarial training based on the first stage of adversarial training using source domain data and target domain data with pseudo labels; S7: Obtain the target data set to be semantically segmented; The target dataset is input into the semantic segmentation model after adversarial training in the second stage to complete the semantic segmentation of the target dataset.

2. The domain adaptation semantic segmentation method integrating adversarial training and self-training according to claim 1, characterized in that: Step S1 includes: S11: Acquire a data set with labels as source domain data, and read a first image picture and a label picture from the source domain data; S12: Crop the first image and the label image into a size of The image is normalized. S13: Obtain the data set to be predicted as the target domain data, read the second image picture from the target domain data; crop the second image picture into The image is normalized and the Normalize operation is performed.

3. The domain adaptation semantic segmentation method integrating adversarial training and self-training according to claim 1, characterized in that: Step S2 includes: Build a semantic segmentation model; Use the downsampled part of the semantic segmentation model as the feature extractor G; Use the upsampled part of the semantic segmentation model as a classifier ; Classifier-based Introduce Gaussian distribution to randomize the convolution weights to form a random classifier ; In the initialization of the semantic segmentation model, instantiate a feature extractor G and a classifier and two random classifiers , .

4. The domain adaptation semantic segmentation method integrating adversarial training and self-training as claimed in claim 3, characterized in that: Step S3 includes: S31: The preprocessed source domain data Input feature extractor G, feature extractor G outputs feature ; S32: Source domain features Input random classifiers respectively , , , according to the predicted probability output by the three classifiers , , Calculate segmentation loss ; S33: By minimizing the segmentation loss To optimize feature extraction G and , , : in is the size of the source domain image, Represents the number of categories of source domain data, is the true label value corresponding to the source domain data; S34: Repeat steps S31 to S33 until the set maximum number of iterations is reached.

5. The domain adaptation semantic segmentation method integrating adversarial training and self-training as claimed in claim 3, characterized in that: Step S4 includes: S41: Input the preprocessed source domain data and target domain data into the feature extractor G to obtain output source domain features respectively , target domain features ; The source domain features Input classifier and a random classifier , The predicted probability , , , calculate the segmentation loss , and optimize G, , , ; S42: Target domain features Input Random Classifier , The predicted probability , ,calculate , The difference loss : in Represents the number of categories of target domain data, and the difference loss is given by and It consists of two parts. represents the joint classification deterministic loss function, represents the local classification deterministic loss function; By maximizing the difference loss Optimize the feature extractor G; S43: By minimizing the difference loss Optimizing random classifiers , ,as follows: The target domain features Input Random Classifier , Get the new predicted probability , and the new difference loss , continue to calculate and minimize the difference loss To continuously optimize the random classifier , ; S44: Take the predicted probability obtained in step S41 , , calculate the predicted probability and predicted probability The average value is taken as the centralized prediction result of the random classifier on the source domain data ; The dimension is The probability value and dimensions are The probability value Concatenate along the last dimension to get a dimension of Tensor ; right The last dimension of The probability and the following The probability and of the bits are combined to obtain a shape of Tensor ; The tensor As source domain features Field label ; The last dimension has Bit; S45: Take the predicted probability calculated in step S43 , The average value is used as the centralized prediction result of the random classifier on the target domain data ; The target domain features Input classifier Get the predicted probability ; according to , get the target domain features Field label ; S46: Use 1 to represent data from the source domain and 0 to represent data from the target domain to create a real domain label ,calculate , and The sum of the cross loss entropy is used as the domain loss : in, Represents source domain data; Represents the target domain data; S47: By minimizing the domain loss Optimizing the classifier , Random Classifier , Random Classifier , and feature extractor G.

6. The domain adaptation semantic segmentation method integrating adversarial training and self-training as claimed in claim 1, characterized in that: Step S5 includes: S51: traverse each batch of target domain data in the target domain data loader, each batch contains only one image; S52: Use the adversarially trained semantic segmentation model to extract and predict features for each image; The random classifier of the semantic segmentation model predicts the features of each image k times and takes the average as the final prediction result; The probability map is calculated by upsampling and normalizing the final prediction result; By performing the maximum value operation on each pixel of the probability map, the confidence value of each pixel belonging to different categories is obtained; Store each confidence value in the category's confidence dictionary; S52: sorting the confidence list of each category in the confidence dictionary in descending order, and for each category, taking the value of the eightieth percentile position from the sorted list as the category threshold of the category; S53: traverse the target domain data loader, and load the corresponding probability map for each batch of target domain data; The final prediction result of each image is divided by the category threshold to calculate the weighted probability and determine the category label of each pixel in the image; pixels with a weighted probability less than 1 are marked as 255, and 255 is set as an ignorable label and does not participate in the calculation of the loss function; S54: The category label of each pixel in the image is saved as a pseudo label in the form of an image in a specified path.

7. The domain adaptation semantic segmentation method integrating adversarial training and self-training as claimed in claim 5, characterized in that: Step S6 includes: S61: re-read the target domain data containing the pseudo-labels, so that the target domain data loader contains the target domain data and the corresponding pseudo-labels; S62: Input the source domain data and the target domain data with pseudo labels into the feature extractor G to obtain the output source domain features respectively. , target domain features ; The source domain features Input classifier and a random classifier , The predicted probability , , , calculate the source domain segmentation loss , optimize the feature extractor G and classifier ; S63: Calculate the difference loss in the same way as step S42 , by maximizing the difference loss To optimize the feature extractor; S64: Same as step S43 by minimizing the difference loss For random classifier , Perform k optimizations; S65: Same as step S44 and step S45, calculate the integrated probability of the prediction results of the random classifier for the source domain data and the target domain data , , calculate the corresponding domain label prediction and domain loss ; Use the cross entropy loss function to calculate the target domain prediction results Segmentation loss with pseudo labels : in Represents the target domain data, Represents the number of categories of target domain data, is the pseudo label corresponding to the target domain data; By minimizing the target domain segmentation loss and minimize the domain loss , for a random classifier , Optimize by minimizing the target domain segmentation loss and maximize the domain loss , optimize the feature extractor G.

8. An electronic device, characterized in that: The electronic device comprises a processor (501), a memory (505), a user interface (503) and a network interface (504), wherein the memory (505) is used to store instructions, the user interface (503) and the network interface (504) are used to communicate with other devices, and the processor (501) is used to execute the instructions stored in the memory (505) so that the electronic device executes the method according to any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed by a computer, the method according to any one of claims 1 to 7 is executed.

Images