Multi-Core Lock-Free Rate Limiting via Virtual Queues
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current multi-core rate limiting methods face scalability issues and high processor overhead due to lock contention and frequent interruptions, especially in software-based implementations, which hinder efficient high-bandwidth network traffic processing and QoS management in SDN and NFV environments.
Innovation Solution
The implementation of a multi-core lock-free rate limiting framework using virtual QoS class queues that isolate simultaneous access by different processor cores, with demand rate and supply rate values to synchronize QoS constraints, eliminating the need for semaphores and reducing lock contention.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional multi-core rate limiting methods are used, then processor overhead is reduced, but scalability is limited due to lock contention and frequent interruptions
Solution Approach 1:
The patent segments the shared queue into multiple per-core queues, where each processor core has its own dedicated queue. This eliminates lock contention by allowing each core to independently manage its own queue without needing to acquire locks, thereby improving scalability and reducing processor overhead while maintaining rate limiting functionality.
2Adaptability or versatility
If software-based rate limiting is implemented, then flexibility is improved, but processor overhead increases due to frequent interruptions
Solution Approach 1:
Each processor core independently manages its own per-core queue without requiring interrupts or synchronization with other cores. The rate limiting logic is distributed across cores, allowing each core to self-service its queue operations without external intervention, thereby reducing processor overhead and interrupt frequency while maintaining software-based flexibility.
3Reliability
If locks and semaphores are used for queue access control, then data consistency is ensured, but processor load increases due to lock contention
Solution Approach 1:
The patent eliminates the need for locks and semaphores by segmenting the shared queue into independent per-core queues. Each core has exclusive access to its own queue, ensuring data consistency without requiring synchronization primitives. This segmentation approach maintains reliability while eliminating the processor load associated with lock contention.
Data Source
Figure 1
Figure 2
Figure 3A~3C
AI summary
An apparatus, such as a network element, comprises a receiver to receive a plurality of packets. A memory stores instructions and forms a first and second set of virtual queues to store the plurality of packets. A processor having one or more cores with one or more packet classifiers provides a classification of a packet in the plurality of packets. The processor in communication with the memory executes instructions to transfer the packet from the receiver to a virtual queue in the first set of virtual queues based on the classification. The processor also transfers the packet from the virtual queue to a transmitter based on a demand rate value and supply rate value associated with the virtual queue.