Lock-Free Scalable Free List for Multi-Processor Systems

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing free list data structures in multiprocessor systems experience significant L2 processor cache misses, even with atomic, lock-free operations, due to inefficient allocation and deallocation of resources like memory, leading to performance issues.

Innovation Solution

Maintaining processor-specific free list data structures with ordered queues and a global free list, where items are dynamically reallocated between local and global lists based on thresholds to optimize resource distribution and prevent cache misses, incorporating a primary and secondary free list pair mechanism for load-balancing.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If a singly-linked list of free items is used in a multi-processor system, then memory allocation and deallocation can be performed, but significant L2 processor cache misses occur leading to performance degradation

Engineering Contradiction:
Improvememory allocation performanceVSAvoidL2 processor cache misses
Core Design Contradiction:
ProductivityVSLoss of time

Solution Approach 1:

The patent divides the single global free list into multiple processor-specific free lists, where each processor has its own dedicated list. This segmentation localizes memory access to each processor's L1 cache, eliminating L2 cache misses caused by shared list access while maintaining efficient allocation and deallocation operations.

Inventive Principle:
Principle #1Segmentation

2Loss of time

If processor-specific free lists are maintained for each processor, then L2 cache misses are reduced, but system complexity increases due to multiple data structures

Engineering Contradiction:
Improvecache miss reductionVSAvoidfree list data structure management
Core Design Contradiction:
Loss of timeVSDevice complexity

Solution Approach 1:

The patent combines multiple processor-specific free lists with a global free list into a unified hierarchical structure. When local lists are empty, processors can allocate from the global list, and excess items can be transferred to the global list for redistribution. This merging approach maintains the cache performance benefits of local lists while reducing overall system complexity through a fallback mechanism.

Inventive Principle:
Principle #5Merging (Combining)

3Adaptability or versatility

If items are dynamically transferred between free lists based on thresholds, then load balancing is improved, but additional operations increase processing overhead

Engineering Contradiction:
Improveload balancingVSAvoidthreshold-based item transfer operations
Core Design Contradiction:
Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent uses threshold parameters to trigger automatic item transfers between free lists. When the number of items in a processor-specific list exceeds an upper threshold, excess items are transferred to the global list. When below a lower threshold, items are replenished from the global list. This parameter-based approach provides adaptive load balancing with simple, efficient operations that minimize processing overhead.

Inventive Principle:
Principle #35Parameter changes

Data Source

PatentUS9892031B2Lock-free scalable free list
Publication Date: 2018.02.13 SAP SE
  • US9892031B2 patent drawing
  • US9892031B2 patent drawing
  • US9892031B2 patent drawing

AI summary

A plurality of free list data structures are maintained in a multi-processor computing system that each correspond to one processor of the multi-processor computing system and that each comprise an ordered queue of processor-specific items. Thereafter, a number of processor-specific items allocated to each free list data structure is calculated. Processor-specific items allocated to a first of the free list data structures are moved to a second of the free list data structures when the number of calculated processor-specific items in the first free data structure exceeds a first threshold. In addition, processor-specific items allocated to the second of the free list data structures are moved to the first of the free list data structures when the number of calculated processor-specific items in the first free data structure is below a second threshold. Related apparatus, systems, techniques and articles are also described.