Multi-Tasking Neural Network Training with Partial Labels
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current deep learning models require separate training for each task, leading to inefficiencies in maintenance and storage, and multi-task learning struggles with insufficient labeled data across multiple tasks.
Innovation Solution
A method for training a multi-tasking network using partly labeled datasets, employing semisupervised learning for tasks without labels and supervised learning for tasks with labels, by calculating task losses and consistency losses to generate a total loss for training, allowing the network to perform multiple tasks efficiently.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If multiple separate deep learning models are trained for each task, then each task can be performed with dedicated training data, but the number of models increases leading to maintenance difficulties and storage inefficiency
Solution Approach 1:
The patent merges multiple separate deep learning models into a single multi-task learning model that can perform multiple tasks simultaneously. This is achieved by sharing common layers and task-specific layers within one unified model architecture, reducing the total number of models from multiple separate models to one integrated model that handles all tasks.
Solution Approach 2:
The patent creates a universal deep learning model capable of performing multiple different tasks through a single model. The model achieves multi-functionality by incorporating task-specific branches that can be selectively activated based on the input task, allowing one model to replace multiple specialized models.
2Device complexity
If a single multi-task learning model is trained, then maintenance and storage efficiency improve, but insufficient labeled data for some tasks degrades training quality
Solution Approach 1:
The patent introduces an intermediary mechanism in the form of task-specific layers that act as mediators between the shared common layers and the final task outputs. These intermediary layers allow the model to adapt shared representations to specific task requirements, enabling effective learning even when labeled data for certain tasks is limited by leveraging knowledge from other tasks through the shared architecture.
3Reliability
If all datasets are labeled for all tasks, then complete supervision improves training accuracy, but labeling time and cost become impractical
Solution Approach 1:
The patent applies partial supervision by training the multi-task model using only the subset of tasks for which labeled data is available, rather than requiring complete labels for all tasks. The model leverages the labeled portions effectively through the shared architecture while handling unlabeled tasks through transfer learning and knowledge sharing from labeled tasks, achieving practical training without exhaustive labeling.
Data Source
AI summary
There is provided a method for training a multi-tasking network performing multi-tasks by using datasets having different task labels. In response to acquiring specific training data from main dataset including 1-st sub dataset having 1-st task label to n-th sub dataset having n-th task label, a learning device inputs the specific training data into a 1-st multi-tasking network to an n-th multi-tasking network, to thereby instruct the 1-st multi-tasking network to the n-th multi-tasking network to perform learning operation on the specific training data and to output n task results; calculates a 1-st task loss to an n-th task loss by referring to 1-st specific task result to n-th specific task result; calculates a 1-st unlabeled consistency loss group to an n-th unlabeled consistency loss group; and trains the 1-st multi-tasking network to the n-th multi-tasking network by using a total task loss and a total consistency loss.