Lock-Free Skiplist Scheduler Queue for Bulk Priority Insertion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional scheduling queue architectures face processing bottlenecks and inefficiencies in job prioritization due to increased thread contention and the inability to implement global job prioritization across multiple CPU cores.
Innovation Solution
A lock-free priority-driven job queue structured as a skiplist allows for efficient bulk insertion of jobs by multiple threads, using sub-skiplists generated in local memory without affecting other threads, and dynamically adjusts the number of queues based on job load to optimize performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a single scheduling queue is used with mutex-based access control, then global job prioritization is maintained, but processing bottlenecks occur due to thread contention
Solution Approach 1:
The patent divides the single scheduling queue into multiple per-CPU scheduling queues, allowing different CPU cores to access their own queues independently. This segmentation eliminates mutex contention between threads while maintaining scheduling control through a hierarchy where per-CPU queues feed into a global scheduling structure.
Solution Approach 2:
The patent introduces a lock-free skiplist data structure as an intermediary layer between per-CPU queues and the global scheduling system. This skiplist enables efficient priority-based job retrieval without requiring traditional mutex locks, acting as a mediator that preserves global prioritization semantics while eliminating contention bottlenecks.
2Productivity
If multiple scheduling queues are used (one per CPU core), then thread contention is reduced, but global job prioritization cannot be implemented
Solution Approach 1:
The patent implements a nested queue structure where per-CPU scheduling queues are nested within a global skiplist hierarchy. Each CPU has its own queue, but these queues are organized in a nested structure that allows higher-level global priority management. Jobs can be inserted into per-CPU queues locally while the nested skiplist structure ensures global prioritization is maintained across all CPUs.
Solution Approach 2:
The patent adds a hierarchical dimension to the scheduling queue structure by organizing per-CPU queues into a multi-level skiplist. This dimensional transformation allows the system to simultaneously support local per-CPU autonomy (reducing contention) and global priority control (maintaining reliability) by operating at different levels of the hierarchy.
3Reliability
If traditional lock-based queue access is used, then thread synchronization is ensured, but processing delays increase due to cache invalidations and thread interference
Solution Approach 1:
The patent replaces the mechanical lock-based synchronization system with a lock-free algorithmic approach using atomic operations and the skiplist data structure. Instead of using physical mutex locks that cause cache invalidations and thread blocking, the system uses atomic compare-and-swap operations on the skiplist structure, eliminating the need for heavy-weight locking mechanisms while maintaining synchronization guarantees.
Solution Approach 2:
The patent changes the fundamental parameter of thread synchronization from lock-based mutual exclusion to lock-free atomic operations. By transforming the synchronization mechanism from a blocking, cache-coherency-dependent approach to an atomic operation-based approach, the system reduces processing delays caused by cache invalidations while maintaining thread synchronization through different computational parameters.
Data Source
Figure 1
Figure 2A~2C
Figure 3
AI summary
A system includes identification of a plurality of computing jobs associated with a same execution priority, determination of an insertion point for each one of a second plurality of queues of a second skiplist based on the execution priority, each of the second plurality of queues including a queue entry for at least one of a second plurality of computing jobs, generation of a first skiplist including a first plurality of queues, the first skiplist including one or more queue entries for each of the plurality of computing jobs, and insertion of each of the first plurality of queues of the first skiplist into a respective one of the second plurality of queues at a respective one of the determined insertion points.