Job Group Scheduling for Semaphore-Limited Data Management
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing job scheduling methods in data management systems lead to delays and underutilization of resources due to semaphores being full, resulting in inefficient job dispatching and reduced throughput.
Innovation Solution
Grouping jobs into job groups based on shared semaphores, allowing each group to be scheduled independently by separate dispatchers, thereby avoiding iteration over full semaphores and minimizing resource underutilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a single dispatcher schedules all jobs sequentially, then the scheduling logic is simple, but job delays increase and throughput decreases when semaphores are full
Solution Approach 1:
The patent divides jobs into multiple groups based on their semaphore dependencies, with each group assigned to a dedicated dispatcher. This segmentation allows parallel scheduling of independent job groups, eliminating the sequential bottleneck of single-dispatcher systems and enabling simultaneous progress across multiple semaphore pools without increasing overall system complexity
2Ease of operation
If a single dispatcher iterates over all jobs to check semaphore availability, then job assignment is centralized, but resource underutilization occurs and scheduling efficiency decreases
Solution Approach 1:
The patent segments the centralized scheduling function into multiple distributed dispatchers, each responsible for a specific job group. This eliminates the need for single dispatcher to iterate over all jobs and check semaphore availability across the entire system, reducing scheduling time and preventing resource underutilization while maintaining manageable job assignment through group-level coordination
Solution Approach 2:
The patent introduces a dimensional shift from single-point centralized scheduling to multi-point distributed scheduling across different dispatcher threads. This dimensional expansion allows simultaneous semaphore availability checks and job assignments across multiple dimensions (parallel dispatcher threads), transforming the sequential bottleneck into parallel processing opportunity
3Reliability
If jobs wait for semaphore availability in a single dispatcher system, then job execution is coordinated centrally, but delays accumulate and throughput is limited
Solution Approach 1:
The patent segments the job queue into multiple independent groups, each with its own dispatcher that can immediately begin scheduling jobs as semaphores become available. This eliminates the sequential waiting period inherent in single-dispatcher systems, as multiple dispatchers operate simultaneously and independently, reducing overall job delay while maintaining coordinated execution through group-based semaphore management
Solution Approach 2:
The patent enables continuous job scheduling by maintaining multiple dispatchers that can immediately pick up and execute jobs as semaphores become available, rather than having a single dispatcher idle while waiting for semaphore availability. This continuous action across multiple parallel dispatchers eliminates delays and maximizes throughput while preserving reliable coordination through shared semaphore resources
Data Source
AI summary
Methods, systems, and devices for data management are described. A data management system (DMS) may provide backup and recovery services for customer computing systems or databases, which may involve scheduling jobs to perform the backup and recovery services. Each job may define a set of semaphores which may be acquired prior to execution of the job. The semaphores may be representative of an availability of computing resources associated with the DMS. Jobs may be grouped into job groups based on the semaphores associated with each job. Each job group may be scheduled independently by separate dispatchers or job schedulers. Within a job group, the associated dispatcher(s) may schedule jobs if the semaphore(s) associated with the job group are available. Within a job group, the associated dispatchers may refrain from dispatching additional jobs if the semaphore(s) for the job group are full until the semaphore(s) are available.