Friend-training Cross-Task Neural Network Pseudo-label Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current self-training algorithms primarily focus on single tasks and datasets, failing to leverage shared properties of inputs across related tasks, limiting their effectiveness in improving model performance through cross-task supervision.
Innovation Solution
The proposed method, 'friend-training,' maps pseudo-labels from different but related tasks into a shared space, computes a matching score, and uses an augmented selector to choose high-quality pseudo-labels based on both matching scores and model confidences, enabling cross-task neural network model training.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If self-training algorithms are applied to single tasks and datasets, then model performance can be improved through pseudo-labels, but the ability to leverage shared properties across related tasks is lost
Solution Approach 1:
The patent combines multiple self-training algorithms operating on different but related tasks into a unified cross-task self-training framework. The system merges pseudo-labels from multiple tasks by mapping them to a shared label space and computing matching scores, allowing the model to leverage shared properties across tasks while maintaining task-specific performance improvements
Solution Approach 2:
The patent creates a universal self-training framework that can handle multiple different task types simultaneously. The cross-task self-training algorithm is designed to be task-agnostic, working with any related tasks that share underlying properties, making the system versatile across different NLP tasks while maintaining reliable performance improvement
2Adaptability or versatility
If multiple self-training algorithms are combined for cross-task learning, then shared properties can be leveraged, but algorithm complexity increases
Solution Approach 1:
The patent introduces several intermediary components to manage the complexity of combining multiple self-training algorithms: (1) a label mapping mechanism that translates pseudo-labels from different tasks into a shared space, (2) a matching score computation module that quantifies agreement between algorithms, and (3) an augmented selector that intelligently chooses which pseudo-labels to use. These intermediaries simplify the integration process while enabling cross-task learning
Solution Approach 2:
The patent segments the cross-task self-training process into distinct modular components: individual task-specific self-training modules, a label mapping module, a matching score computation module, and a pseudo-label selection module. This segmentation allows each component to be developed and optimized independently, reducing overall algorithmic complexity while maintaining the ability to leverage multiple tasks
Data Source
AI summary
Method, apparatus, and non-transitory storage medium for training two or more cross-task neural network models based on two or more neural network tasks, including mapping first pseudo labels based on a first model associated with a first task among the two or more neural network tasks and second pseudo labels based on a second model associated with a second task among the two or more neural network tasks to a same space, and computing a matching score indicating a cross-task matching between the first pseudo labels and the second pseudo labels based on the mapping. The method may further include selecting one or more cross-task pseudo labels based on the matching score and accuracies associated with the first model and the second model, and training the two or more cross-task neural network models based on the one or more cross-task pseudo labels.