A source domain noise correction method for unsupervised domain adaptation

Abstract

This invention provides a source domain noise correction method for unsupervised domain adaptation, belonging to the field of machine vision domain adaptation. The method first obtains all samples X in the source domain. S The category is N. The feature extractor and classifier are pre-trained in the source domain. The pre-trained feature extractor is then used to extract the feature set Z from all samples in the source domain. S The extracted feature set Z was processed using a Gaussian mixture model (GMM) with K=N cluster categories. S The method involves modeling, calculating and comparing the probability of each sample x belonging to each cluster, and updating the label of the current sample by taking the cluster label of the cluster with the highest probability. Then, the number of samples is counted using the label transformation matrix A, and the original label of the sample with the highest number of samples corresponding to the new label is used as the corrected label for all samples corresponding to the new label, thus completing the correction of source domain noise. This invention reclassifies the noisy source domain dataset according to its maximum probability value, reducing the noise ratio of the dataset and enhancing the robustness of the unsupervised domain adaptation method.

Description

A Source Domain Noise Correction Method for Unsupervised Domain Adaptation Technical Field

[0001] This invention belongs to the field of machine vision domain adaptation, specifically relating to a source domain noise correction method for unsupervised domain adaptation. Background Technology

[0002] Unsupervised domain adaptation is a transfer learning problem where the source domain is labeled and the target domain is unlabeled. The goal is to transfer knowledge from a supervised learning model in the source domain to the target domain, achieving a perfect fit. However, existing unsupervised domain adaptation methods almost entirely rely on large, clean datasets. In reality, noisy images are inevitable during data collection, such as labeling errors caused by human error or the use of web crawlers to scrape large amounts of noisy web images. Research shows that noisy samples increase model complexity, make training more difficult, and ultimately reduce model accuracy. If noisy samples reach a certain proportion, their impact on the model can be devastating. Therefore, an accurately labeled dataset is crucial for supervised learning in the source domain, which is typically ensured by noise correction of the source domain dataset.

[0003] In existing technologies, some researchers choose to use manual annotation and noise removal methods, but this approach is time-consuming, labor-intensive, and extremely costly. Taking image segmentation as an example, image segmentation requires very accurate pixel-level annotation. For such image data, manual annotation would take an average of 1.5 hours to ensure annotation quality, resulting in high time and cost. Other researchers use weakly supervised learning methods for dataset annotation. Their approaches to handling sample noise mainly fall into two categories: one is to reduce the impact of noisy samples through noisy learning to enhance the model's robustness and thus improve performance; the other is to use relevant algorithms to clean up noise and remove noisy samples from the dataset.

[0004] Among noisy learning methods, improving the loss function is the best way to effectively reduce the impact of noisy samples. However, this approach is ultimately a temporary solution, and noisy samples still affect model performance. Researchers, through rigorous mathematical derivation and extensive experiments, have found that loss functions with symmetry can completely negate the effects of symmetric noise. However, in reality, most naturally generated noise samples during dataset collection are asymmetric, and improving the loss function becomes less effective when dealing with naturally generated noise. The most direct method for noise removal is to delete outlier samples with significant impact. This type of method typically involves a feature extractor extracting features from the source domain samples, followed by a classifier deriving the probability values ​​of the samples in each category. Based on this, samples with abnormal probability values ​​are identified and processed, and the more appropriate the processing method, the better the effect. For example, a filtering voting method removes a sample only when most learners in the set agree to remove it. However, this method can lead to the accidental deletion of some correct samples, resulting in "over-cleaning," causing the deletion of some correct instances, missing key data, decreased model generalization ability, and ultimately reduced domain adaptation performance. Summary of the Invention

[0005] In view of the above-mentioned defects or deficiencies in the prior art, the present invention aims to provide a source domain noise correction method for unsupervised domain adaptation. The method uses a truncated Gaussian mixture model to model the source domain dataset, and corrects the noise in the dataset through algorithm comparison, thereby achieving scientific and effective noise correction, improving the unsupervised domain adaptation effect, enhancing the robustness and universality of the model, and improving the overall performance of the model.

[0006] To achieve the above objectives, the embodiments of the present invention adopt the following technical solutions:

[0007] A source domain noise correction method for unsupervised domain adaptation includes the following steps:

[0008] Step S1: Obtain all samples X from the source domain. S and the original label set Y of all samples S And transform each sample x into tensor form, and the source domain samples X... S The number of categories is N;

[0009] Step S2: Pre-train the feature extractor and classifier in the source domain, and use the pre-trained feature extractor to extract the feature set Z from all samples in the source domain. S ;

[0010] Step S3: Set the number of clusters in the Gaussian Mixture Model (GMM) to K, where K = N; apply the Gaussian Mixture Model to the extracted feature set Z. SModel the data, calculate and compare the probability that the current sample x belongs to each cluster, use the cluster label of the cluster with the highest probability as the new label for prediction, and update the label of the current sample.

[0011] Step S4, construct a label transition matrix A = {a} of size N×N. ij}, a ij Let a represent the number of samples whose original label is (j-1) and whose predicted new label is (i-1). Calculate all elements a in matrix A. ij The value;

[0012] Step S5: Based on the statistics of the number of samples in matrix A, the original label of the sample with the most samples corresponding to the new label is used as the corrected label for all samples corresponding to the new label, thus completing the correction of the source domain noise.

[0013] In a preferred embodiment of the present invention, step S4 calculates a ij The process is as follows:

[0014] Set each a ij The initial value is a ij = 0, where the value of (i-1) represents the new label predicted by the Gaussian mixture model for the current sample, and the value of (j-1) represents the original label of the current sample. All labels take values ​​of 0, 1, ..., N-1. When the new label and the original label of the sample correspond to (i-1) and (j-1) respectively, the corresponding a ij =a ij +1; Iterate through every sample in the source domain to obtain matrix A.

[0015] As a preferred embodiment of the present invention, the pre-training described in step S2 enables the feature extractor and classifier to achieve a stable accuracy on the test set.

[0016] In a preferred embodiment of the present invention, the stable accuracy rate has a fluctuation range below a predetermined threshold.

[0017] In a preferred embodiment of the present invention, the predetermined threshold is 0.1%.

[0018] In a preferred embodiment of the present invention, step S3 calculates the probability that the current sample x belongs to each cluster. The formula for calculating the probability P(x) that each sample x belongs to all clusters is as follows:

[0019] P(x)=(p 1 (x), p 2 (x), ..., p k (x), ..., p K (x)),

[0020] pk (x)=α k N(x|μ k , ∑ k (1)

[0021] In formula (1), p k (x) represents the probability that sample x belongs to the k-th cluster, α k The weights of the Gaussian distribution representing the k-th cluster are derived from the internal algorithm of the Gaussian mixture model; N(x|μ k , ∑ k ) represents a Gaussian distribution, μ k Let ∑ represent the mean of the k-th cluster. k Let μ represent the covariance matrix of the k-th cluster. k and ∑ k All are derived from the Gaussian mixture model, and:

[0022]

[0023] Substituting formula (2) into formula (1) yields p. k (x).

[0024] In a preferred embodiment of the present invention, step S5 specifically includes:

[0025] Step S51: The initial value of c is preset to 0, and the initial value of t is preset to 1;

[0026] Step S52: Query the column containing the maximum value in the t-th row of the label transformation matrix A, where m is the column number.

[0027] Step S53: If m ≠ t, swap the t-th row with the m-th row so that the maximum value of the m-th row is on the diagonal of the matrix, and increment c by 1; if m = t, then increment c by 1 and t by 1.

[0028] Step S54: If c < N-1, return to step S52; if c = N-1, proceed to step S55.

[0029] Step S55: Use the new labels (i-1) corresponding to all elements in the transformed matrix A′ as the corrected labels for all corresponding samples in the source domain.

[0030] The technical solutions provided in the embodiments of the present invention have the following beneficial effects:

[0031] The source domain noise correction method for unsupervised domain adaptation provided in this embodiment of the invention uses GMM for feature modeling and first obtains all samples X in the source domain. S and the original label set Y of all samples S And transform each sample x into tensor form, and the source domain samples X... SThe number of classes is N; the feature extractor and classifier are pre-trained in the source domain, and the pre-trained feature extractor is used to extract the feature set Z from all samples in the source domain. S Set the number of GMM clusters to K, where K = N; use a Gaussian mixture model to process the extracted feature set Z. S Modeling is performed to obtain the probability that each sample x belongs to each cluster, and the probability of the current sample in all clusters is compared. The cluster label of the cluster with the highest probability is taken to update the label of the current sample; then an N×N label transition matrix A = {a ij}, any element a in the matrix ij The number of samples with the original label (j-1) predicted to have the new label (i-1) is calculated. Based on the statistics of sample counts in matrix A, the original label corresponding to the sample with the largest number of samples with the new label is used as the corrected label for all samples corresponding to that new label, thus completing the correction of source domain noise. This invention, as a clustering method, effectively reclassifies noisy datasets in the source domain according to their maximum probability values, reducing the proportion of noise in the dataset and minimizing the impact of noise on the source domain model. It enhances the robustness of unsupervised domain adaptation methods without affecting the model's generalization ability.

[0032] Of course, implementing any product or method of the present invention does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description

[0033] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0034] Figure 1 is a flowchart of the source domain noise correction method for unsupervised domain adaptation provided in an embodiment of the present invention;

[0035] Figure 2 is a schematic diagram of the label transformation matrix A obtained in an application example of an embodiment of the present invention;

[0036] Figure 3 shows the transformation diagram of the label transformation matrix A obtained in the application example of this embodiment of the invention;

[0037] Figure 4 is a schematic diagram of the transformed label transformation matrix A′ in an application example of this invention. Detailed Implementation

[0038] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. It should be noted that, without conflict, the embodiments and features in the embodiments of the present invention can also be combined with each other.

[0039] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In the description of this invention, the terms "first," "second," "third," "fourth," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0040] Unsupervised domain adaptation methods aim to transfer the model to the target domain and adapt it well by learning from existing supervised source domain data in an unsupervised environment. However, in real-world scenarios, most source domain data contains label noise, which significantly impacts model accuracy. This invention provides a source domain noise correction method for unsupervised domain adaptation. To reduce the impact of noise on the source domain model, a "gentle" noise processing approach is adopted, without removing any data. The correction method first uses a convolutional network to extract features from the source domain data. Then, a Gaussian Mixture Model (GMM) is used to model the extracted features. GMMs have clustering effects, allowing the generation of probability values ​​for different sample features under different categories, thus identifying potential noise samples. Finally, based on the obtained probability values, a truncated Gaussian method is used to adjust or leave unprocessed potential noise samples, ultimately achieving a scientifically effective noise correction and improving the overall model performance.

[0041] Referring to Figure 1, the source domain noise correction method for unsupervised domain adaptation provided in this embodiment of the invention includes the following steps:

[0042] Step S1: Obtain all samples X from the source domain. S and the original label set Y of all samples S And transform each sample x into tensor form, and the source domain samples X... S The sample size is M, and the number of categories is N.

[0043] In this step, the entire sample X of the source domain S This includes noise samples, which can be obtained through any means. It is precisely because of the total number of samples X in the source domain... SNoise samples in the dataset can affect the subsequent domain adaptation effect. This embodiment does not use the original method of deleting noise samples or improving the loss function of the learning process, but instead uses the method of correcting noise samples, that is, without weakening the integrity of the sample set, while improving the unsupervised domain adaptation effect.

[0044] Step S2: Pre-train the feature extractor and classifier in the source domain, and use the pre-trained feature extractor to extract the feature set Z from all samples in the source domain. S .

[0045] In this step, the pre-training enables the feature extractor and classifier to achieve a stable accuracy on the test set. Preferably, the accuracy fluctuation range is below a predetermined threshold. The predetermined threshold is 0.1%.

[0046] Step S3: Set the number of clusters in the Gaussian Mixture Model (GMM) to K, where K = N; apply the Gaussian Mixture Model to the extracted feature set Z. S The model is constructed by calculating and comparing the probability that the current sample x belongs to each cluster, using the cluster label of the cluster with the highest probability as the new label for prediction, and updating the label of the current sample; this process is repeated for all samples, updating the original label set Y. S Generate a new label set Y G .

[0047] In this step, the probability P(x) that each sample x belongs to all clusters is calculated using the following formula:

[0048] P(x)=(p 1 (x), p 2 (x), ..., p k (x), ..., p K (x)),

[0049] p k (x)=α k N(x|μ k , ∑ k (1)

[0050] In formula (1), p k (x) represents the probability that sample x belongs to the k-th cluster, α k The weights of the Gaussian distribution representing the k-th cluster are derived from the internal algorithm of the Gaussian mixture model; N(x|μ k , ∑ k ) represents a Gaussian distribution, μ k Let ∑ represent the mean of the k-th cluster. k Let μ represent the covariance matrix of the k-th cluster. k and ∑ k All are derived from the Gaussian mixture model, and:

[0051]

[0052] Substituting formula (2) into formula (1) yields p. k (x).

[0053] Step S4, construct a label transition matrix A = {a} of size N×N. ij}, where N is the number of classes in all samples of the source domain, a ij Let a represent the number of samples whose original label is (j-1) and whose predicted new label is (i-1). Calculate all elements a in matrix A. ij The value of .

[0054] In this step, a is calculated. ij The process is as follows:

[0055] Set each a ij The initial value is a ij = 0, where the value of (i-1) represents the new label of the current sample obtained by the Gaussian mixture model, and the value of (j-1) represents the original label of the current sample among all samples in the source domain. All labels take values ​​of 0, 1, ..., N-1. When the new label and original label of a sample correspond to (i-1) and (j-1) respectively, the corresponding a ij =a ij +1; Iterate through every sample in the source domain to obtain matrix A. In fact, matrix A is a counting matrix that counts the number of samples that changed from the same original label to the same new label.

[0056] If the value of a certain element in matrix A is very large, it means that the original labels of a certain number of samples in a certain category of all samples in the source domain have been transformed into the same new label. In this case, even though the new label of the current sample predicted by GMM is wrong, the clustering of the corresponding cluster can still be considered correct.

[0057] Step S5: Based on the statistics of the number of samples in matrix A, the original label of the sample with the most samples corresponding to the new label is used as the corrected label for all samples corresponding to the new label, thus completing the correction of the source domain noise.

[0058] This step specifically includes:

[0059] Step S51: The initial value of c is preset to 0, and the initial value of t is preset to 1;

[0060] Step S52: Query the column containing the maximum value in the t-th row of the label transformation matrix A, where m is the column number.

[0061] Step S53: If m ≠ t, swap the t-th row with the m-th row so that the maximum value of the m-th row is on the diagonal of the matrix, and increment c by 1; if m = t, then increment c by 1 and t by 1.

[0062] Step S54: If c < N-1, return to step S52; if c = N-1, proceed to step S55.

[0063] Step S55: Use the new labels (i-1) corresponding to all elements in the transformed matrix A′ as the corrected labels for all corresponding samples in the source domain. Continue domain adaptation using the labeled data to ultimately improve domain adaptation performance.

[0064] The following is a specific practical example for illustration.

[0065] This application example uses the MNIST digit dataset as the noisy source domain and the USPS digit dataset as the target domain. The source domain digit dataset contains N = 10 classes (0-9) and 60,000 image samples. The first 10,000 images are used as the source domain test set and not participated in training, while the last 50,000 images are used as the source domain training set with 40% labeled noise. The target domain digit dataset contains 10 classes (0-9) and 9,298 image samples, all of which are used as test samples for the target domain accuracy after domain adaptation. Taking the adversarial discriminative domain adaptation method as an example, the acquired source domain samples X... S In the data, 40% of the tag noise is pair-flip noise under pair flipping conditions, denoted as P40.

[0066] The training set samples from the source domain digit dataset are loaded and fed into the LeNet network for pre-training. Because the training set contains 40% noise, the model's accuracy on the test set is only 67.30% after N training rounds. The training set samples from the source domain dataset are loaded again, and the feature extractor from the pre-trained model extracts the feature set Z for all samples in the source domain. S It contains 50,000 features. A Gaussian Mixture Model (GMM) is declared, and the number of clusters to be clustered is set to K = 10. The GMM is then applied to the feature set Z of the entire training set. S Clustering is performed on the features in the data. The probability of each sample belonging to each cluster is obtained according to formulas (1) and (2). The probability of each sample in all clusters is compared. The cluster label of the cluster with the highest probability is taken to update the label of the current sample, which is used as the new label of the current sample predicted by GMM.

[0067] Construct a 10×10 matrix A. Each element in matrix A is obtained from the original labels of the dataset and the new labels predicted by the GMM for all samples. As shown in Figure 2, the first row represents the samples whose new label predicted by the GMM is 0. There are 14 samples with an original label of 0, 11 samples with an original label of 1, and the most samples with an original label of 4, totaling 2864. The other rows are similar.

[0068] The maximum value in row 1 is 2864 in column 5, indicating that the 2864 samples with a label of 4 in the entire source domain were predicted by GMM and the resulting cluster label was 0. The data in row 1 and row 5 are swapped, as shown in Figure 3. At this point, column 5 of row 5 contains the maximum value, and the "new label" is corrected to 4, the same as the original label with the most samples. This new label is then used as the corrected label for all samples corresponding to row 5.

[0069] At this point, the first row is the original fifth row. Using this row as the first row again, repeat step S51, which involves finding the row number corresponding to the maximum value in the first row, which is now the eighth column. Then, swap the first and eighth rows. Continue this process until the maximum values ​​of all rows are on the diagonal, resulting in a new matrix A′, as shown in Figure 4.

[0070] Using the new label (i-1) corresponding to matrix A′ at this point as the corrected label for the corresponding sample in the source domain, we continue to perform domain adaptation using the data with the corrected label, ultimately improving the domain adaptation performance. Continuing to perform unsupervised domain adaptation using the corrected sample set can significantly reduce the impact of noisy samples on domain adaptation. This is applied to all samples X in the source domain, including two different types of noise. S Source domain noise correction was performed, and the accuracy before and after correction was compared, as shown in Table 1. This application example corresponds to the data in the same column as P40 in Table 1, and also includes related experiments with different noise levels at different ratios, all listed in Table 1. Wherein, S20 represents a 20% proportion of symmetrical flip noise under symmetrical flipping; S40 represents a 40% proportion of symmetrical flip noise under symmetrical flipping; and P20 represents a 20% proportion of paired flip noise under paired flipping.

[0071] Table 1

[0072]

[0073] As shown in Table 1, source domain samples with noise levels below 40% were corrected, resulting in a significant decrease in the label noise ratio of the source domain. Subsequent pre-training using the corrected source domain samples significantly improved accuracy on the source domain test set, greatly reducing the impact of noisy source domain samples on the subsequent domain adaptation process, enhancing the robustness of the domain adaptation model, and improving the performance of unsupervised domain adaptation models in noisy source domain conditions.

[0074] As can be seen from the above technical solutions, the source domain noise correction method for unsupervised domain adaptation provided in this embodiment of the invention uses GMM for feature modeling, effectively reclassifies the noisy source domain dataset according to its maximum probability value, reduces the noise ratio of the dataset, and enhances the robustness of the unsupervised domain adaptation method.

[0075] The above description is merely a preferred embodiment of the present invention and an explanation of the technical principles employed, and is not intended to limit the scope of the claimed invention, but merely to illustrate preferred embodiments of the invention. Those skilled in the art should understand that the scope of the invention is not limited to the specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the inventive concept. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

Claims

1. A source domain noise correction method for unsupervised domain adaptation, characterized in that, It includes the following steps: Step S1, obtain all the image samples X of the source domain S and the original label set Y of all samples S , and transform each sample x into a tensor form. The number of categories in all samples X of the source domain S is N; Step S2, pre-train the feature extractor and classifier on the source domain, and use the pre-trained feature extractor to extract the feature set Z S from all samples of the source domain; Step S3, set the number of clusters in the Gaussian mixture model GMM to K, and K = N; use the Gaussian mixture model to model the extracted feature set Z S , calculate and compare the probability that the current sample x belongs to each cluster, use the cluster label of the cluster with the maximum probability as the predicted new label, and update the label of the current sample; Step S4, construct a label transformation matrix A of size N×N = {a ij}, a ij represents the number of samples with the original label (j - 1) and the predicted new label (i - 1), and calculate the values of all elements a ij in the matrix A; Step S5, according to the statistics of the number of samples in the matrix A, use the original label corresponding to the sample with the largest number of samples corresponding to the new label as the corrected label for all samples corresponding to the new label, and complete the correction of the source domain noise; where, Step S5 specifically includes: Step S51, preset the initial value of c to 0 and the initial value of t to 1; Step S52, query the column m where the maximum value in the t-th row of the label transformation matrix A is located; Step S53, if m≠t, swap the t-th row and the m-th row to make the maximum value in the m-th row on the diagonal of the matrix, and c is incremented by 1; if m = t, then c is incremented by 1 and t is incremented by 1; Step S54, if c < N - 1, return to Step S52; if c = N - 1, execute Step S55; Step S55, use the transformed matrix A The new label (i-1) corresponding to all elements in the source domain is used as the correction label for all corresponding samples in the source domain.

2. The source domain noise correction method for unsupervised domain adaptation according to claim 1, characterized in that, In step S4, calculate a ij The process is as follows: Set each a ij The initial value is a ij =0, where the value of (i-1) represents the new label predicted by the Gaussian mixture model for the current sample, and the value of (j-1) represents the original label of the current sample. All labels take values ​​of 0, 1, ..., N-1. When the new label and the original label of the sample correspond to (i-1) and (j-1) respectively, the corresponding a ij =a ij +1; Iterate through every sample in the source domain to obtain matrix A.

3. The source domain noise correction method for unsupervised domain adaptation according to claim 1, characterized in that, The pre-training described in step S2 enables the feature extractor and classifier to achieve stable accuracy on the test set.

4. The source domain noise correction method for unsupervised domain adaptation according to claim 3, characterized in that, The stable accuracy rate is defined as an accuracy rate whose fluctuation range is below a predetermined threshold.

5. The source domain noise correction method for unsupervised domain adaptation according to claim 4, characterized in that, The predetermined threshold is 0.1%.

6. The source domain noise correction method for unsupervised domain adaptation according to claim 1, characterized in that, In step S3, the probability that the current sample x belongs to each cluster is calculated. The formula for calculating the probability P(x) that each sample x belongs to all clusters is as follows: (1) In formula (1), Let x represent the probability that sample x belongs to the k-th cluster. The weights of the Gaussian distribution of the k-th cluster are derived from the internal algorithm of the Gaussian mixture model. Indicates a Gaussian distribution. This represents the mean of the k-th cluster. Let the covariance matrix of the k-th cluster be denoted as . and All are derived from the Gaussian mixture model, and: (2) Substituting formula (2) into formula (1) yields the solution. 。

Images