Learning Apparatus for Domain Adaptation via Distribution Loss
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In defective image classification using machine learning, changes in product generation, such as shape or manufacturing process changes, can lead to different defect types, causing a shift in data distribution. This can result in lower classification precision when using domain adaptation methods, especially when pseudo-labels are incorrect.
Innovation Solution
A learning apparatus that acquires training data without teaching labels, generates estimation vectors through a model, calculates a distribution loss between the estimation distribution and a target distribution using KL divergence, and updates the model parameters based on this loss. This process improves classification precision even without labeled data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If domain adaptation is used to combine source domain and target domain training data, then classification precision for the target domain is improved, but if the class distribution assumption is different between source and target domains, the classification precision in the target domain becomes lower
Solution Approach 1:
The patent changes the learning objective from classification accuracy to distribution matching. By minimizing the difference between source domain label distribution and target domain label distribution, the method adapts to different class distributions without requiring identical assumptions, thereby resolving the contradiction between improving classification precision and handling distribution differences.
Solution Approach 2:
The patent introduces an intermediary objective function (distribution difference minimization) that bridges the source domain and target domain. This intermediary approach allows the model to adapt to target domain data characteristics without directly optimizing for target domain classification, thus handling distribution mismatches effectively.
2Measurement precision
If pseudo-label method is used for domain adaptation, then classification precision may be improved, but if the pseudo-label is incorrect, incorrect learning is performed and it is difficult to improve the classification precision
Solution Approach 1:
The patent extracts and removes the problematic element (pseudo-labels) from the domain adaptation process. Instead of generating potentially incorrect pseudo-labels for target domain data, the method directly optimizes for distribution matching between source and target domains, eliminating the source of incorrect learning while maintaining the ability to improve classification precision.
3Productivity
If a trained model from old generation is used for new generation products, then previous training data can be utilized, but because of product generation changes, the classification precision becomes lower
Solution Approach 1:
The patent performs preliminary adaptation by minimizing distribution differences between source and target domains before actual classification tasks. This preliminary distribution alignment ensures that when the adapted model is applied to new generation products, it maintains high classification precision while effectively reusing previous training data.
Data Source
AI summary
According to one embodiment, a learning apparatus includes a processor. The processor acquires first training data. The processor inputs the first training data to a model, and generate a plurality of estimation vectors that are a processing result of the model. The processor generates an estimation distribution from the estimation vectors. The processor calculates a distribution loss between the estimation distribution and a target distribution that is a target in an inference using the model. The processor updates parameters of the model, based on the distribution loss.