Multitask Machine Learning Framework With Adaptive Parameter Sharing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing multitask learning architectures face challenges with limited training data and performance degradation due to dissimilarities between pooled prediction classes, which limits the predictive capabilities of machine learning models.
Innovation Solution
A new machine learning framework that jointly trains models based on inferred similarities between them, using a similarity matrix and loss functions to optimize predictive performance across models, allowing for intelligent sharing of parameters and iterative refinement of similarities.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If machine learning models are jointly trained based on task similarities, then predictive capabilities are improved, but performance is damped by unrelated aspects of the similar tasks
Solution Approach 1:
The patent applies local quality by allowing different levels of parameter sharing based on the specific similarity between tasks. Instead of uniformly sharing all parameters across similar tasks, the system selectively shares only those parameters that are relevant to each specific task pair, thereby improving predictive capabilities while avoiding performance damping from unrelated task aspects.
Solution Approach 2:
The patent implements dynamics by making the parameter sharing mechanism adaptive rather than static. The system dynamically adjusts which parameters to share and to what extent based on the measured similarity between tasks, allowing the model to flexibly optimize for each task while still benefiting from joint training on related tasks.
2Productivity
If machine learning models are trained with minimal training data, then training efficiency is improved, but predictive accuracy is insufficient
Solution Approach 1:
The patent applies universality by creating a joint training framework that serves multiple tasks simultaneously with a shared parameter space. This allows the model to learn common patterns across tasks from minimal data per task, improving training efficiency while maintaining or enhancing predictive accuracy through the shared knowledge representation.
Solution Approach 2:
The patent merges multiple task-specific models into a unified joint training framework where parameters are shared across tasks. By combining the training objectives and parameter spaces of multiple tasks, the system achieves better predictive accuracy with minimal training data per task, as the shared parameters learn robust representations that generalize across tasks.
3Quantity of substance
If machine learning parameters are shared across models, then training data requirements are reduced, but model specificity is lost
Solution Approach 1:
The patent applies segmentation by dividing parameters into task-specific and shared components. This allows the model to maintain specificity for each task through task-specific parameters while still benefiting from data efficiency through shared parameters. The segmentation enables selective sharing of only those parameters that benefit from joint training without compromising task-specific performance.
Solution Approach 2:
The patent implements local quality by allowing different degrees of parameter sharing for different tasks based on their similarity. Highly similar tasks share more parameters to reduce data requirements, while less similar tasks maintain more task-specific parameters to preserve model specificity. This localized approach to parameter sharing optimizes both data efficiency and task-specific performance.
Data Source
AI summary
Various embodiments of the present disclosure disclose a machine learning training approach for intelligently training a plurality of machine learning models associated with a multitask environment. The techniques include jointly training the plurality of machine learning models based on task similarities by generating a similarity matrix corresponding to a plurality machine learning models, generating a sharing loss value for the at least two machine learning models, generating, using a loss function and a training dataset, a prediction loss value for a particular machine learning model of the at least two machine learning models, generating an aggregated loss value for the particular machine learning model based on the similarity matrix, the sharing loss value, and the prediction loss value, and updating the particular machine learning model based on the aggregated loss value for the particular machine learning model.