Per-CPU Reference Counting for NUMA Locking Bottlenecks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In multi-threaded systems, locking mechanisms used for reference counting lead to unacceptable overhead and bottlenecks, especially in non-uniform memory architectures (NUMA) and non-uniform memory access (NUMA) systems, impacting processing times and increasing the likelihood of processors being in waiting states, particularly in time-critical and processor-intensive applications.
Innovation Solution
Implement per-CPU reference counting that leverages per-CPU operations such as interrupt handling, pre-emption, and inter-process interrupts to avoid complex synchronization primitives, using per-CPU counters for increment and decrement operations, and transitioning to an atomic state for resource reclamation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If locking mechanisms are used for reference counting in multi-threaded systems, then resource sharing and reclamation can be ensured, but locking overhead becomes unacceptable and processing performance deteriorates
Solution Approach 1:
The patent divides the reference counting system into per-CPU counters instead of using a single shared counter. Each CPU maintains its own reference count, eliminating the need for locks when incrementing or decrementing counts on the same CPU. This segmentation allows parallel operations across different CPUs without synchronization overhead, resolving the contradiction between reliability and productivity.
Solution Approach 2:
The patent introduces per-CPU operations and interrupt handlers as intermediaries to manage reference counting. Instead of direct lock-based synchronization, the system uses CPU-specific operation queues and interrupt mechanisms to coordinate reference count updates, reducing locking overhead while maintaining correctness through controlled transition points.
2Productivity
If per-CPU reference counting is implemented, then locking overhead is reduced and performance improves, but complex synchronization primitives are still needed for coordination
Solution Approach 1:
The patent extracts the synchronization complexity from the reference counting operations themselves and concentrates it into specific per-CPU interrupt handlers and transition mechanisms. By isolating the coordination logic to these specialized components rather than embedding it in every reference count operation, the system reduces overall complexity while maintaining performance benefits.
Solution Approach 2:
The patent implements dynamic state transitions between per-CPU mode and atomic mode based on system conditions. The reference counting mechanism can switch between using per-CPU counters for normal operations and atomic operations for critical transitions, optimizing the balance between performance and coordination complexity adaptively.
3Reliability
If atomic operations are used for reference counting, then thread safety is ensured, but the likelihood of processors being in waiting states increases
Solution Approach 1:
The patent applies different quality levels of synchronization to different locations in the system. Per-CPU counters provide local thread safety without requiring global atomic operations, eliminating processor waiting states for same-CPU operations. Atomic operations are reserved only for cross-CPU coordination where they are truly necessary, minimizing overall waiting time while maintaining thread safety.
Data Source
AI summary
Per-CPU reference counting leveraging per-CPU operations is presented herein. An example method comprises receiving a read request for access to shared data from a thread executing on one processor of a multi-processor core, determining that the shared data is unavailable in a first cache memory, transmitting the read request to storage server equipment, polling a second cache memory to determine that the shared data is unavailable in the second cache memory, based on the unavailability of shared data in the first cache memory and second cache memory, sending the shared data to the first cache memory, providing access to the shared data, incrementing a counter value, and based on the shared data having been modified, writing the modified shared data to the first cache memory for future access to the modified shared data by the thread.