Lock-Free Skiplist Scheduler Queue for Bulk Job Insertion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional scheduling queue architectures face processing bottlenecks and inefficiencies in job prioritization due to increased thread contention, particularly in multi-core systems, leading to substantial processing delays and reduced scalability.
Innovation Solution
A lock-free priority-driven job queue structured as a skiplist allows for efficient bulk insertion of jobs by multiple threads, utilizing a sub-skiplist generated in local memory and inserted atomically into the active skiplist without affecting other threads, and dynamically adjusts the number of queues based on job load to optimize performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a single scheduling queue is used to manage queue contention, then queue access is simplified, but wait times for accessing the queue become a processing bottleneck as the number of available threads increases
Solution Approach 1:
The scheduling queue is segmented into multiple sub-queues (one per CPU core), allowing concurrent access by multiple threads without contention. Each thread can independently access its assigned sub-queue, eliminating the need for mutex locks and reducing wait times while maintaining simplified access mechanisms.
2Productivity
If multiple scheduling queues are managed (e.g., one per CPU core) to reduce contention, then processing throughput improves, but the ability to implement global job prioritization is impacted because threads cannot retrieve jobs from other queues
Solution Approach 1:
Multiple sub-queues are merged into a unified priority-based scheduling structure where jobs from any sub-queue can be selected for execution based on global priority. The scheduler maintains awareness of all sub-queues and can retrieve high-priority jobs from any CPU core's queue, combining the benefits of reduced contention with global prioritization.
3Productivity
If bulk insertion of jobs is performed by multiple threads simultaneously, then job processing efficiency improves, but cache invalidations and thread interference increase
Solution Approach 1:
The job insertion process is segmented by assigning each thread its own sub-queue in local memory. Threads can independently insert jobs into their local sub-queues without interfering with other threads' cache lines. This eliminates cache invalidations while maintaining bulk insertion efficiency, as each thread operates on dedicated memory regions.
Data Source
AI summary
A system includes identification of a plurality of computing jobs associated with a same execution priority, determination of an insertion point for each one of a second plurality of queues of a second skiplist based on the execution priority, each of the second plurality of queues including a queue entry for at least one of a second plurality of computing jobs, generation of a first skiplist including a first plurality of queues, the first skiplist including one or more queue entries for each of the plurality of computing jobs, and insertion of each of the first plurality of queues of the first skiplist into a respective one of the second plurality of queues at a respective one of the determined insertion points.