Multitask Learning for Heterogeneous Sparse Datasets
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing multitask learning methods face challenges in learning datasets with different feature spaces and suffer from performance imbalances due to varying data distributions and labeling criteria, leading to overfitting and degraded inference performance.
Innovation Solution
The proposed solution involves a multitask learning apparatus and method that includes a projection layer to align heterogeneous datasets into a common feature space, a shared layer for extracting common features, and an optimization technique to balance learning performance between tasks using data augmentation and adjustment of representation and task-specific losses.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If datasets with different feature spaces are learned using existing multitask learning methods, then learning can be performed on multiple tasks, but the feature vectors cannot be aligned and common features cannot be extracted
Solution Approach 1:
The patent applies parameter changes by introducing a projection layer that transforms feature vectors from different heterogeneous datasets into a unified feature space. This projection layer learns task-specific projection parameters that map diverse input features (images, text, audio) to a common representation space, enabling the shared layer to extract common features across tasks while preserving task-specific characteristics.
Solution Approach 2:
The patent introduces a projection layer as an intermediary component between the input datasets and the shared layer. This intermediary transforms heterogeneous feature vectors into a standardized format that can be processed by the shared layer, facilitating common feature extraction while maintaining the ability to handle diverse data types and feature spaces.
2Productivity
If a shared layer is used to extract common features, then learning efficiency improves, but performance imbalance and overfitting occur when tasks have different data distributions
Solution Approach 1:
The patent implements dynamics by making the contribution of each task to the shared layer adaptive rather than fixed. The projection layer dynamically adjusts the weighting and transformation of each task's features based on its importance and data distribution characteristics. This allows the system to automatically balance the influence of different tasks during training, preventing overfitting to dominant tasks while still benefiting from common feature extraction.
Solution Approach 2:
The patent applies local quality by allowing each task to have its own projection parameters and loss function characteristics while sharing the common feature extraction layer. This enables the system to tailor the learning process to each task's specific requirements (local optimization) while still benefiting from shared representations (global efficiency), thereby maintaining performance balance across tasks with different distributions.
3Quantity of substance
If data augmentation is applied to heterogeneous datasets, then the amount of training data increases, but the complexity of aligning augmented data across different feature spaces increases
Solution Approach 1:
The patent applies preliminary action by performing data augmentation at the input level before the projection layer processes the data. Augmented samples from different datasets are prepared in advance with their respective augmentations (e.g., image transformations, text variations), and then the projection layer handles the alignment to common feature space. This separation of augmentation and alignment operations simplifies the overall process by dealing with each task's data characteristics independently before unification.
Data Source
AI summary
Provided are a multitask learning apparatus and method for improving learning performance of heterogeneous small datasets. The multitask learning apparatus includes a first layer configured to generate feature vectors by projecting training data pairs generated from different tasks to one feature space, a second layer configured to extract a common feature from the projected feature vectors, and a third layer configured to draw each individual inference from the extracted common feature. Here, the first layer and the third layer are task-specific layers, and the second layer is a layer shared between tasks. The first layer, the second layer, and the third layer perform forward propagation in one artificial neural network.