Lockless Timer Management for Multi-Threaded Queue Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Multi-threaded applications face performance degradation due to the need for locks to safely share data blocks/structures between threads, which imposes substantial system performance costs, especially when moving items to the back of the queue requires a lock on the entire queue, causing all threads to wait.
Innovation Solution
A lockless timers system and method that allow updating timers associated with multi-threads without using locks, using a virtual managing entity to divert virtual objects between queues based on their TTL and TIQ, and allocating access time based on the number of threads and TIQ, thereby minimizing the number of queues and optimizing queue management.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If locks are used to safely share data blocks/structures between threads, then data sharing safety is improved, but system performance deteriorates due to substantial performance costs and thread waiting
Solution Approach 1:
The patent segments the queue management by creating multiple independent queues (e.g., head queue, tail queue, middle queues) instead of using a single shared queue. Each queue can be managed independently by different threads without requiring global locks, thus maintaining data safety while improving performance. The segmentation allows parallel access to different parts of the queue structure.
Solution Approach 2:
The patent introduces an intermediary mechanism (such as lock-free data structures, comparison-swaps, or intermediate buffer queues) that mediates between threads accessing shared resources. This intermediary allows threads to coordinate access without requiring traditional locks, eliminating the performance penalty while ensuring safe data sharing through atomic operations or buffered communication.
2Stability of the object's composition
If a lock is placed on the entire queue to move items to the back, then queue data integrity is improved, but all threads must wait causing performance degradation
Solution Approach 1:
The queue is segmented into multiple independent sections (head queue, tail queue, middle queues) that can be accessed and modified independently. Threads can add items to the tail queue or remove items from the head queue without requiring locks on the entire queue structure, eliminating global waiting while maintaining data integrity through localized access control.
Solution Approach 2:
The patent implements self-service mechanisms where threads can independently perform queue operations (enqueue, dequeue, move operations) using lock-free algorithms or atomic operations. Each thread manages its own access to specific queue portions without requiring external locking, allowing concurrent operations without mutual exclusion and eliminating thread waiting time.
3Productivity
If multiple queues are used to manage virtual objects, then thread concurrency is improved, but system complexity increases
Solution Approach 1:
The system segments virtual objects into different queues based on their characteristics (e.g., TTL values, priority levels, time intervals). This segmentation enables parallel processing of different object types while maintaining manageable queue structures. Each queue handles specific object categories, reducing the complexity of managing all objects in a single structure while improving concurrency.
Solution Approach 2:
The patent implements dynamic queue management where the number and configuration of queues can adapt based on system conditions, object characteristics, and concurrency requirements. Queues can be dynamically created, merged, or reconfigured without system shutdown, allowing the system to optimize for concurrency while keeping complexity manageable through adaptive structuring rather than fixed complex architecture.
Data Source
AI summary
A method for executing instructions embedded in two threads stored in a system including two operating units and a virtual managing entity for holding queues for virtual objects (VO) waiting to use a respective operating unit and diverting them between queues. Each VO is associated with two virtual timers, one measuring a time period during which the VO is held in the queue (TIQ) and the other providing time period during which the VO will remain in an alive state (TTL). The method includes receiving information relating to VOs associated with the two threads; operating on VOs for which their TTLs have expired; upon emerging from its respective queue, determining whether each VO should be diverted to another queue; upon diverting the VO, resetting its TIQ timer; and allocating an access time to each VO based on a number of threads requiring that VO and the TIQ associated therewith.