Multi-Task Learning Device Batch Size Calculation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional multi-task learning by stochastic gradient descent is ineffective when learning data includes samples not fully tagged with labels for all tasks, as it lacks an algorithm to determine appropriate batch sizes, leading to incomplete learning.
Innovation Solution
A learning device that calculates batch sizes for each task such that the data size divided by the batch size is consistent across tasks, allowing for appropriate sampling and weight updates using stochastic gradient descent.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional multi-task learning by stochastic gradient descent is used with learning data that is not fully tagged, then learning cannot be appropriately performed, but adding lacking labels to samples requires an enormous amount of work
Solution Approach 1:
The patent changes the parameter of batch size from a fixed uniform value to task-specific variable values. By calculating different batch sizes for different tasks based on their respective data sizes, the system enables appropriate learning completion without requiring additional labeling work. The batch size for each task is determined by dividing the task's data size by a common iteration count, allowing each task to be processed appropriately despite varying data completeness.
2Ease of operation
If one batch size is determined for a plurality of tasks, then task-by-task sampling can be repeated with the same batch size, but learning data varies in data size when samples are not fully tagged
Solution Approach 1:
The patent segments the batch size parameter by task, allowing each task to have its own optimized batch size rather than using a single uniform batch size for all tasks. This segmentation enables the system to handle varying data sizes across tasks while maintaining consistent iteration counts, resolving the conflict between sampling consistency and data size variation adaptability.
Solution Approach 2:
The patent introduces dynamic batch size determination where the batch size for each task is calculated based on the task's specific data size and a common iteration count. This dynamic approach allows the batch size to adapt to each task's characteristics while maintaining operational consistency through the unified iteration framework.
3Reliability
If the data size of learning data varies from task to task, then appropriate learning cannot be performed without an algorithm for determining batch sizes, but uniform batch size determination is simple
Solution Approach 1:
The system performs self-service by automatically calculating appropriate batch sizes for each task based on their data sizes and a common iteration count. Rather than requiring manual configuration or complex external algorithms, the system autonomously determines the optimal batch size for each task, ensuring appropriate learning while keeping the implementation relatively simple through automatic computation.
Data Source
AI summary
A learning data processing unit accepts, as input, a plurality of pieces of learning data for a respective plurality of tasks, and calculates, for each of the tasks, a batch size which meets a condition that a value obtained by dividing a data size of corresponding one of the pieces of learning data by the corresponding batch size is the same between the tasks. A batch sampling unit samples, for each of the tasks, samples from corresponding one of the pieces of learning data with the corresponding batch size calculated by the learning data processing unit. A learning unit updates a weight of a discriminator for each of the tasks, using the samples sampled by the batch sampling unit.