Transfer Learning Layer Selection by Domain Data Richness
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Transfer learning in machine learning often disregards domain properties, leading to reduced inference precision and unnecessary process volume growth.
Innovation Solution
Adapt the model by selectively training layers of the target model based on domain adaptation data richness, using a neural network model like VGG16, to maintain high precision and generalization capability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If transfer learning is performed by disregarding domain properties, then the process is simple, but the inference precision and generalization quality are lowered
Solution Approach 1:
The patent applies local quality by differentiating the treatment of different network layers based on their specific functions and the domain adaptation data richness. Instead of uniformly training all layers, the system selectively trains only certain layers (e.g., higher layers for rich data, lower layers for poor data) according to their specific roles in feature extraction and domain adaptation, thereby improving inference precision while maintaining operational simplicity.
2Adaptability or versatility
If all layers of the target model are trained, then the model adapts well to the target domain, but the volume of process grows unnecessarily large
Solution Approach 1:
The patent segments the neural network layers into different groups based on their function and the domain adaptation data richness. By dividing the layers into trainable and non-trainable groups, the system achieves effective domain adaptation without the computational burden of training all layers, thus reducing process volume while maintaining adaptability.
Solution Approach 2:
The patent applies partial action by training only the necessary portion of layers rather than all layers. Depending on the domain adaptation data richness, the system selectively trains either higher layers (for rich data) or lower layers (for poor data), performing just enough training to achieve good domain adaptation without excessive computational processes.
3Productivity
If fewer layers are trained, then the processing volume is reduced, but the generalization capability may be insufficient
Solution Approach 1:
The patent introduces dynamics by making the training scope adaptive rather than fixed. The system dynamically determines which layers to train based on the domain adaptation data richness, allowing the training strategy to flexibly adjust between training more or fewer layers. This dynamic approach ensures sufficient generalization capability while optimizing processing efficiency for each specific scenario.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A domain adaptation data richness determination unit (52) determines, when a first model trained by using training data of a first domain is trained by transfer learning by using training data of a second domain, a domain adaptation data richness based on the number of items of training data of the second domain, the first model being a neural network. A learning layer determining unit (54) determines a layer in the second model, which is a duplicate of the first model, targeted for training, based on the domain adaptation data richness. A transfer learning unit (56) applies transfer learning to the layer in the second model targeted for training, by using the training data of the second domain.