Offload Engine for Hardware-Accelerated Data Lookup
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data lookup operations in network functions, particularly in virtual switches, face performance bottlenecks due to high latency and resource consumption, especially in cloud-native scenarios where massive network traffic and concurrent data structures lead to increased CPU usage and core-to-core communication delays.
Innovation Solution
Offloading hash-table lookups and tree traversals to a lookup engine, which accelerates flow classification by utilizing a last-level cache (LLC) and employing hardware-accelerator features like query distribution, cache locking, and hardware-implemented hash calculators to reduce CPU compute cycles and improve data access latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If data lookup operations are performed using traditional CPU-based methods, then implementation flexibility is maintained, but processing latency increases and throughput decreases
Solution Approach 1:
The patent replaces CPU-based software lookup operations with a hardware lookup engine that performs data structure traversals in hardware. This substitution of mechanical (software) operations with hardware operations dramatically increases lookup speed while offloading the complexity from the CPU to dedicated hardware circuitry.
Solution Approach 2:
The patent introduces a lookup engine as an intermediary component between the CPU and the data structures stored in memory/LLC. This intermediary handles the complex lookup operations, freeing the CPU from these tasks while providing a dedicated pathway for high-speed data retrieval.
2Productivity
If hash-table lookups are offloaded to a lookup engine, then processing throughput increases, but hardware resource consumption increases
Solution Approach 1:
The patent pre-loads flow classification data structures into the Last Level Cache (LLC) before lookup operations begin. This preliminary action ensures that frequently accessed data is already in high-speed cache memory, reducing the need for repeated main memory accesses and lowering overall energy consumption during throughput-intensive operations.
Solution Approach 2:
The patent implements a hybrid architecture where the lookup engine uses a combination of LLC and main memory, with the LLC serving as a high-speed local cache for frequently accessed data. This local quality optimization reduces average access times and energy consumption by keeping hot data locally available while maintaining the ability to access less frequently used data in main memory.
3Quantity of substance
If concurrent data structures are used to handle massive network traffic, then system capacity increases, but core-to-core communication delays increase
Solution Approach 1:
The patent extracts the lookup operation from the CPU core execution path and relocates it to a dedicated hardware lookup engine. This extraction eliminates the need for CPU cores to participate in lookup operations, thereby removing core-to-core communication delays entirely from the data path while maintaining the ability to handle massive concurrent traffic.
Data Source
AI summary
A central processing unit can offload table lookup or tree traversal to an offload engine. The offload engine can provide hardware accelerated operations such as instruction queueing, bit masking, hashing functions, data comparisons, a results queue, and a progress tracking. The offload engine can be associated with a last level cache. In the case of a hash table lookup, the offload engine can apply a hashing function to a key to generate a signature, apply a comparator to compare signatures against the generated signature, retrieve a key associated with the signature, and apply the comparator to compare the key against the retrieved key. Accordingly, a data pointer associated with the key can be provided in the result queue. Acceleration of operations in tree traversal and tuple search can also occur.