Machine Learning Label Distribution Alignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Transductive training in machine learning faces accuracy issues when there is a difference in label distribution between the transfer source and target domains, leading to incorrect transfers and reduced estimation accuracy due to mismatched feature distributions.
Innovation Solution
A machine learning program that estimates a first label distribution for unlabeled training data based on a classification model and an initial label distribution of the target domain, acquires a second label distribution, calculates label weights based on differences, and re-trains the model using weighted labeled and unlabeled data to align feature distributions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If transductive training is performed using labeled data from a transfer source domain, then the model can be trained with available data, but when there is a difference in label distribution between source and target domains, the feature distributions do not coincide, leading to incorrect transfer and reduced accuracy
Solution Approach 1:
The patent applies parameter changes by adjusting the weight of training data based on label distribution differences. Specifically, it calculates a weight for each training data point using the formula: weight = (target_label_distribution / source_label_distribution), where weights greater than 1 amplify underrepresented samples and weights less than 1 suppress overrepresented samples. This reweighting parameter adjustment aligns the feature distributions between source and target domains, resolving the contradiction between training efficiency and estimation accuracy.
2Reliability
If features unique to the transfer source are used to create the model, then the model captures source domain characteristics, but the accuracy of the model in the transfer target domain degrades
Solution Approach 1:
The patent applies local quality by differentiating the treatment of training samples based on their label distribution characteristics. Instead of uniformly processing all data, it assigns different weights to different samples according to their local label distribution properties. This allows the model to adaptively emphasize source domain features where they are reliable while suppressing features that do not generalize to the target domain, thus maintaining source domain reliability while improving target domain adaptability.
3Ease of manufacture
If the label distribution of the transfer source domain is used for training, then training can proceed with available labeled data, but estimation by the machine learning model becomes biased by the source label distribution, reducing estimation accuracy in the target domain
Solution Approach 1:
The patent implements feedback by iteratively adjusting training weights based on the discrepancy between source and target label distributions. The process calculates the weight for each sample, applies it to retrain the model, and evaluates performance on target domain data. This feedback loop continues until the feature distributions are adequately aligned, ensuring that the model learns from source domain data while correcting for distributional biases, thus maintaining training feasibility while improving estimation accuracy.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A machine learning program that causes at least one computer to execute a process, the process includes estimating a first label distribution of unlabeled training data based on a classification model and an initial value of a label distribution of a transfer target domain, the classification model being trained by using labeled training data which corresponds to a transfer source domain and unlabeled training data which corresponds to the transfer target domain; acquiring a second label distribution based on the labeled training data; acquiring a weight of each label included in the labeled training data and the unlabeled training data based on a difference between the first label distribution and the second label distribution; and re-training the classification model by the labeled training data and the unlabeled training data reflected the weight of each label.