On-Chip Completion Cache for InfiniBand Host Channel Adapter Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Infiniband network completion building processes are inefficient due to the need for repeated memory reads and potential cache misses when building completion status for work requests, leading to increased latency and reduced throughput.
Innovation Solution
Implementing a completion cache on-chip in the host channel adapter (HCA) to cache a portion of the work request, allowing for efficient completion building by querying the cache upon receiving a response and using completion cache timers and a credit scheme to manage cache entries and throttle outgoing requests.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If work request information is stored in main memory, then the system can handle multiple work requests, but memory access latency increases and throughput decreases
Solution Approach 1:
The patent segments the storage system into two parts: a small on-chip completion cache for frequently accessed work request information and main memory for bulk storage. This segmentation allows critical completion building operations to access data from the fast on-chip cache while less frequently accessed data remains in main memory, thereby reducing latency for time-critical operations and maintaining high throughput.
Solution Approach 2:
The patent implements preliminary action by pre-loading work request information into the on-chip completion cache before it is needed for completion building. The cache is populated with work request descriptors and associated data in advance, so when a completion needs to be built, the information is already available in the fast on-chip memory, eliminating the need for slow main memory accesses at the critical moment.
2Productivity
If a large completion cache is implemented, then completion building efficiency improves, but the risk of cache overflow and premature evictions increases
Solution Approach 1:
The patent applies parameter changes by dynamically adjusting the cache retention policy based on the state of the work request. Specifically, the system monitors whether a work request has received an acknowledgment and modifies the cache entry lifetime accordingly. Cache entries are retained longer for requests awaiting acknowledgment and evicted sooner for completed requests, optimizing cache utilization and preventing overflow while maintaining high completion building efficiency.
3Loss of time
If work request information is cached on-chip, then completion building latency is reduced, but the device complexity increases
Solution Approach 1:
The patent extracts only the essential work request information needed for completion building and stores it in the on-chip cache, rather than caching entire work request structures. This extraction approach includes only critical fields such as work request identifiers, descriptor pointers, and acknowledgment status, significantly reducing the cache size and complexity while still achieving the latency reduction benefit of on-chip storage.
Data Source
AI summary
A method for optimizing completion building is disclosed. The method involves receiving a work request by a host channel adapter (HCA), caching a portion of the work request in a completion cache in the HCA, wherein the cached portion of the work request includes information for building a completion for the work request, receiving, by the HCA, a response to the work request, querying the completion cache upon receiving the response to the work request to obtain the cached portion of the work request, and building the completion for the work request using the cached portion of the work request, wherein the completion informs a software application of at least a status of the work request as executed by the HCA.