Dynamic Batch Sizing for Computing Task Management
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing computing systems face inefficiencies in managing multiple computing tasks due to limitations in bandwidth and computational resources, leading to high scheduling time overhead and suboptimal performance in terms of response latency and data throughput.
Innovation Solution
A method for managing multiple computing tasks on a batch basis, which involves identifying task types, acquiring scheduling time overhead, determining a batch size using a mapping model, and dividing tasks into batches based on the determined size, allowing for dynamic adjustment of batch size to balance response latency and data throughput.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If multiple computing tasks are processed one by one, then response latency is reduced, but data throughput decreases and scheduling time overhead increases
Solution Approach 1:
The patent implements dynamic batch size adjustment based on task characteristics and system state. The batch size is not fixed but adapts according to the mapping model that considers task type, scheduling time overhead, and resource availability, allowing the system to optimize between processing speed and throughput dynamically
Solution Approach 2:
The patent changes the parameter of batch size to resolve the contradiction. By adjusting the batch size parameter based on task type and scheduling overhead, the system can process multiple tasks in batches when overhead is low (improving throughput) while maintaining acceptable response latency, thus resolving the trade-off between processing one task at a time and batch processing
2Productivity
If batch size is increased to improve data throughput, then scheduling time overhead increases, but response latency may deteriorate
Solution Approach 1:
The patent employs a mapping model that acts as a feedback mechanism. The model learns from historical scheduling data and system performance metrics to determine optimal batch sizes. This feedback loop allows the system to adjust batch sizes to maximize throughput while keeping scheduling overhead within acceptable limits, preventing the deterioration of response latency
3Loss of time
If batch size is increased to reduce scheduling time overhead, then data throughput improves, but response latency increases
Solution Approach 1:
The system dynamically adjusts batch size based on real-time conditions and task characteristics. For tasks requiring low response latency, the batch size is reduced or set to 1, while for throughput-critical tasks, larger batches are used. This dynamic adaptation resolves the contradiction by making batch size a flexible parameter rather than a fixed value
4Productivity
If computing units are fully utilized to process multiple tasks, then productivity improves, but bandwidth resource limitations cause scheduling complexity to increase
Solution Approach 1:
The patent uses the mapping model to determine optimal batch sizes based on task type and system state. This parameter adjustment simplifies scheduling complexity by providing a systematic approach to batch formation, allowing full utilization of computing units without manual complex scheduling decisions
Data Source
AI summary
The present disclosure relates to a method, a device, and a program product for managing multiple computing tasks on a batch basis. A method includes: identifying a task type of the multiple computing tasks in response to receiving a request to use a computing unit in a computing system to perform the multiple computing tasks; acquiring a scheduling time overhead incurred for scheduling the multiple computing tasks for execution by the computing unit; determining, based on the task type and the scheduling time overhead, a batch size for dividing the multiple computing tasks; and dividing the multiple computing tasks into at least one batch based on the batch size. A corresponding device and a corresponding computer program product are provided. With the example implementations of the present disclosure, the batch size for dividing multiple computing tasks can be dynamically determined, so that the performance of the computing system can meet user demands.