Lock-Free Datapath Design for Parallel Storage Arrays
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional storage and networking software fails to optimize performance with modern parallel hardware interfaces in NVME storage devices and RDMA network adapters, leading to bottlenecks from lock acquisition and contention, which are not efficiently addressed by existing solutions like the Storage Performance Development Kit (SPDK).
Innovation Solution
A lock-free software design that allows parallel management operations by scheduling event handlers on specific CPU cores without locks, using producer-consumer queues for inter-processor messaging and adaptive polling to efficiently move work to ideal hardware interfaces, eliminating the need for traditional mutex/semaphore locks and enabling asynchronous operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional lock-based software design is used to manage storage and networking operations, then data consistency and coordination are maintained, but performance bottlenecks occur due to lock acquisition and contention
Solution Approach 1:
The system divides storage and networking operations into separate workqueues assigned to specific CPU cores. Each core has dedicated event handlers that process operations independently without acquiring locks, eliminating contention while maintaining data consistency through structured workqueue management.
Solution Approach 2:
The patent introduces workqueues as intermediary data structures that mediate between different CPU cores and hardware interfaces. These workqueues enable safe data transfer and coordination between cores without requiring lock-based synchronization, thus improving throughput while preserving reliability.
2Productivity
If polling drivers are implemented to eliminate kernel transitions and interrupt latencies, then storage performance improves, but lock contention remains an issue in traditional software designs
Solution Approach 1:
The system implements self-service polling where event handlers on each CPU core independently poll their assigned workqueues and hardware interfaces. This eliminates the need for lock-based coordination during polling operations, allowing storage performance optimization while avoiding lock acquisition delays.
Solution Approach 2:
The patent pre-assigns specific workqueues and hardware interfaces to particular CPU cores before operations begin. This preliminary assignment enables event handlers to directly access assigned resources without lock contention, eliminating lock acquisition time while maintaining proper resource management.
3Adaptability or versatility
If management operations are performed on the high speed datapath, then configuration and tuning capabilities are enabled, but storage and networking traffic is slowed down
Solution Approach 1:
The system segments management operations from high-speed data operations by assigning management tasks to specific event handlers on dedicated CPU cores. This separation allows management operations to execute without interfering with the speed of storage and networking traffic, maintaining both adaptability and performance.
Solution Approach 2:
The patent uses workqueues as intermediary structures that allow management operations to be queued and processed independently from high-speed data paths. This mediation enables configuration and tuning capabilities while preventing management operations from slowing down critical storage and networking traffic.
Data Source
AI summary
Systems and methods (including hardware and software) are disclosed for us in a multi-core, multi-socket server with many RDMA network adapters and NVME solid state drives. One of the features of the subject matter is to optimize the total IO throughput of the system by first replacing software locks with non-interruptible event handlers running on specific CPU cores that own individual software data structures and hardware queues, and second by moving work to that CPU affinity without stalling due to software lock overhead.