Asymmetric Multi-Task Feature Network for Negative Transfer Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing multi-task learning models face challenges such as negative transfer, inefficiency in memory and time usage, and the inability to assume explicit features, particularly when applied to deep neural networks and as the number of tasks increases.
Innovation Solution
The development of an asymmetric multi-task feature learning method that generates a trained model using a parameter matrix and feedback matrix, allowing asymmetric knowledge transfer between tasks, with an objective function that includes a loss function, auto-encoder term, and weight decay regularization to minimize negative transfer and support deep neural networks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the GO-MTL model is used for multi-task learning with common feature sharing, then task learning performance is improved, but negative transfer occurs when incorrectly learned parameter information is transferred between tasks
Solution Approach 1:
The patent applies asymmetry by introducing different regularization terms for different tasks. Specifically, it uses a group lasso regularization term for tasks that should share features and an l2 regularization term for tasks that should not share features. This asymmetric treatment allows the model to selectively share features between tasks based on their compatibility, preventing negative transfer while maintaining the benefits of feature sharing for compatible tasks.
2Object-generated harmful factors
If the AMTL model is used for asymmetric knowledge transfer, then negative transfer is reduced, but the model cannot be properly applied to deep neural networks and memory/time efficiency decreases
Solution Approach 1:
The patent applies local quality by selectively applying different regularization strategies to different task pairs within the overall multi-task learning framework. Instead of treating all tasks uniformly, the model identifies which task pairs are suitable for feature sharing and applies group lasso regularization only to those pairs, while applying l2 regularization to task pairs that are not suitable for sharing. This localized approach enables proper application to deep neural networks while maintaining asymmetric knowledge transfer benefits.
3Adaptability or versatility
If the number of tasks is increased in the AMTL model, then more tasks can be learned simultaneously, but memory and time cost increases quadratically
Solution Approach 1:
The patent applies segmentation by dividing the multi-task learning problem into two distinct parts: tasks that are suitable for feature sharing and tasks that are not. This segmentation is achieved through the use of different regularization terms - group lasso for shareable tasks and l2 for non-shareable tasks. By segmenting the task set in this way, the model avoids the quadratic complexity increase that would occur with uniform asymmetric knowledge transfer across all task pairs, enabling efficient learning with a larger number of tasks.
Data Source
AI summary
A method for generating a trained model is provided. The method for generating a trained model includes: receiving a learning data; generating an asymmetric multi-task feature network including a parameter matrix of the trained model which permits an asymmetric knowledge transfer between tasks and a feedback matrix for a feedback connection from the tasks to features; computing a parameter matrix of the asymmetric multi-task feature network using the input learning data to minimize a predetermined objective function; and generating an asymmetric multi-task feature trained model using the computed parameter matrix as the parameter of the generated asymmetric multi-task feature network.