Job Scheduler Resource Pairing for Concurrent Worker Execution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional job scheduler systems fail to adequately account for resource disparities between jobs, leading to inefficient consumption of technical resources across job workers.
Innovation Solution
A job scheduler system that associates jobs with consumption types and selectively groups them in a job queue for retrieval by job workers, allowing CPU-intensive, memory-intensive, and network-intensive jobs to be executed concurrently, thereby improving resource utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If traditional job scheduler systems queue jobs for retrieval by job workers without considering job disparities, then the system is simple to operate, but resource utilization across job workers becomes inefficient
Solution Approach 1:
The patent applies local quality by classifying jobs into different consumption types (CPU-intensive, memory-intensive, network-intensive, I/O-intensive) based on their specific resource requirements. This allows the job scheduler to treat different jobs differently, assigning them to appropriate workers based on their consumption patterns, thereby improving resource utilization while maintaining operational simplicity through automated classification
Solution Approach 2:
The system changes the parameter of job scheduling by introducing consumption type classification and using historical processing data to determine job characteristics. This parameter change enables the scheduler to make informed decisions about job assignment and concurrent execution, transforming the system from a simple queue to an intelligent resource optimization platform
2Productivity
If jobs are grouped and executed concurrently by the same job worker, then resource utilization improves, but the system complexity increases
Solution Approach 1:
The patent segments the job scheduling process into distinct components: job classification by consumption type, historical data analysis, tuple formation for concurrent execution, and worker assignment. This segmentation allows the system to manage complexity through modular design, where each component handles a specific aspect of the scheduling decision, making the overall complex system manageable and maintainable
Solution Approach 2:
The system performs preliminary actions by analyzing historical processing data before job execution to determine consumption types and form job tuples. This preliminary classification and grouping work is done in advance, allowing the actual execution phase to proceed efficiently without real-time decision-making complexity, thereby balancing resource utilization improvement with system complexity management
3Productivity
If jobs with different consumption types are executed concurrently, then technical resources are optimized, but job execution coordination becomes more difficult
Solution Approach 1:
The patent applies local quality by creating job tuples that group jobs with complementary consumption types (e.g., CPU-intensive with I/O-intensive) that can execute concurrently without resource conflicts. This localized grouping strategy optimizes technical resource utilization by ensuring that concurrent jobs do not compete for the same resources, while the automated tuple formation process maintains ease of operation by removing manual coordination requirements
Data Source
AI summary
Methods, systems, and computer-readable storage media for receiving a first set of historical processing data for a first job, the first set of historical processing data including a first value for a first characteristic type and a second value for a second characteristic type, determining a first consumption type for the first job based on the first value and the second value, assigning the first job and a second job to a job tuple in response to determining that the first consumption type of the first job is combinable with a second consumption type of the second job, fetching, by a first job worker, the job tuple from a job queue, and concurrently executing, by the first job worker, the first job and the second job of the job tuple.