Separable Transfer Learning Neural Networks for Imbalanced Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing artificial neural networks face challenges in improving performance when faced with imbalanced data or insufficient data for learning, and there is a lack of quantified guidelines for transfer learning to enhance their performance effectively.
Innovation Solution
The proposed system employs separable transfer learning, where two pre-trained neural networks are trained independently on different phase tasks using separate datasets, and their parameters are transferred to a target neural network for fine-tuning, optimizing the phase layer regions to enhance performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If transfer learning is applied to improve artificial neural network performance, then prediction accuracy can be enhanced, but the complexity of the training process increases and overfitting may occur
Solution Approach 1:
The patent segments the artificial neural network into multiple phase layers (first phase layer, second phase layer, etc.) that can be independently trained and transferred. Each phase layer processes different aspects of the learning task, allowing selective transfer of parameters from source to target networks without transferring the entire network, thus reducing training complexity while maintaining accuracy improvement.
Solution Approach 2:
The patent applies local quality by treating different phase layers with different training strategies and parameter transfer approaches. Each phase layer can have its parameters independently adjusted, frozen, or transferred based on its specific characteristics and the requirements of the target task, rather than applying a uniform approach to the entire network.
2Measurement precision
If conventional transfer learning is used, then performance improvement may be achieved, but there is no quantified guideline for when and how to apply it effectively
Solution Approach 1:
The patent introduces a feedback mechanism that evaluates the effectiveness of parameter transfer from source to target networks. By monitoring performance metrics during the transfer learning process, the system can determine whether further parameter transfer is beneficial or if the target network should be trained independently, providing a quantified guideline for when to apply transfer learning.
Solution Approach 2:
The patent systematically varies parameters such as the number of phase layers, the depth of each layer, and the proportion of parameters to transfer, establishing quantified guidelines for optimal configuration. This allows practitioners to select appropriate transfer learning strategies based on specific task requirements rather than trial and error.
3Measurement precision
If multiple neural networks are trained independently on different datasets, then feature extraction capability improves, but training time and computational resources increase
Solution Approach 1:
The patent applies preliminary action by pre-training multiple source neural networks on different datasets before the target task. These source networks extract diverse features that are then transferred to the target network, which requires less training time than training a network from scratch on the target dataset, thus reducing overall training time while maintaining feature extraction capability.
Solution Approach 2:
The patent merges the parameter sets from multiple pre-trained source networks into the target network. By combining parameters from different source networks that have learned different features, the target network benefits from diverse feature representations without requiring each source network to be trained to full convergence, reducing total training time.
Data Source
AI summary
Disclosed herein is a computing system using machine learning-based artificial intelligence. The computing system includes at least one processor, a first artificial neural network configured to pre-train on a first phase task based on a first dataset, a second artificial neural network configured to pre-train on a second phase task based on a second dataset, and a target artificial neural network configured to train on a target task based on a target dataset. The at least one processor transfers the parameter of the first phase layer of the first artificial neural network to the first phase layer of the target artificial neural network, transfers the parameter of the second phase layer of the second artificial neural network to the second phase layer of the target artificial neural network, and performs control so that the target artificial neural network performs fine-tuning training on the target task.