HPC Job Scheduling for Batch Throughput and Interactive Response
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies have not effectively addressed the need for interactivity in high performance computing (HPC) systems, specifically in the technical problem of interactivity, with short time lags from the viewpoint of user's request and operability, and processing performance, interactive processing, and system, in the technical problem of addressing the need for interactivity in existing high performance computing (HPC) systems, particularly in the context of batch processing methods and gang scheduling techniques, which often result in suboptimal performance for jobs requiring immediate user interaction and varying interactivity needs across different time zones.
Innovation Solution
A job execution control method and device that dynamically allocates time resources by alternating between batch and interactive processing, prioritizing interactive jobs when resources are tight, and ensuring that jobs are executed efficiently by determining the tightness of time resources using various indices, such as overhead rates and maximum delay times, to improve overall system responsiveness and efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If batch processing is prioritized to improve processing performance, then productivity increases, but interactivity and responsiveness to user requests deteriorate
Solution Approach 1:
The system dynamically adjusts the scheduling strategy based on real-time resource availability and job characteristics. When resources are abundant, batch processing is prioritized for high productivity. When resources are tight or interactive jobs arrive, the system switches to prioritize interactive jobs, ensuring responsiveness. This dynamic adaptation resolves the contradiction between batch processing efficiency and interactive responsiveness.
Solution Approach 2:
The system changes the scheduling parameters (priority levels, time allocation) based on system state. Interactive jobs can be assigned higher priority parameters when needed, while batch jobs receive optimized time slots. This parameter adjustment allows the system to maintain high productivity for batch jobs while ensuring interactivity when required.
2Productivity
If gang scheduling is used to improve batch processing efficiency, then productivity increases, but time lag for interactive jobs increases
Solution Approach 1:
The system segments the scheduling into different queues or priority levels: batch processing queues and interactive job queues. Gang scheduling is applied to batch jobs to maximize efficiency, while interactive jobs are handled separately with lower time lags. This segmentation allows both objectives to be achieved simultaneously without one adversely affecting the other.
Solution Approach 2:
The system implements periodic checking of resource availability and job priorities. At scheduled intervals, it evaluates whether to continue batch processing or switch to handling interactive jobs. This periodic action ensures that batch processing efficiency is maintained while preventing excessive time lags for interactive jobs by switching context when appropriate.
3Ease of operation
If equal time allocation is given to batch and interactive jobs, then fairness is improved, but overall system responsiveness deteriorates
Solution Approach 1:
The system implements asymmetric time allocation where interactive jobs receive preferential treatment with shorter waiting times and higher priority, while batch jobs receive longer time slots optimized for efficiency. This asymmetric approach is fair in the sense that it matches resource allocation to job requirements: interactive jobs need quick response, batch jobs need throughput. This resolves the contradiction by defining fairness differently - not as equal time, but as appropriate time for each job type.
Solution Approach 2:
The system allocates excessive time resources to batch jobs when possible, allowing them to run efficiently without interfering with interactive jobs. Interactive jobs are guaranteed minimum responsive allocation. This partial allocation strategy ensures system responsiveness for interactive jobs while maximizing batch processing efficiency when resources permit.
Data Source
AI summary
A non-transitory computer-readable recording medium stores therein a job execution control program that causes a computer to execute a process including receiving a first job for performing batch processing and a second job for performing interactive processing with a user, determining tightness of a time resource used to cause a predetermined calculation node to alternately execute the first job and the second job according to a lapse of time;in a case where the time resource is not tight, allocating an equal time to the first job and the second job and first causing the predetermined calculation node to make first execution, and in a case where the time resource is tight, increasing priority of execution of the second job, allocating a time to the first job and the second job, and second causing the predetermined calculation node to make second execution.