Distributed Queue Pair State Cache for InfiniBand Host Channel Adapter
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In Infiniband networks, quickly switching between queue pairs for simultaneous communication by multiple processes is challenging due to the need for rapid access and management of queue pair states, which are currently not efficiently cached and distributed across hardware components.
Innovation Solution
A distributed cache system on the host channel adapter (HCA) is implemented, where each module has a sub-cache block to store specific portions of the queue pair (QP) states, allowing efficient access and management of QP states through a centralized logic for caching and eviction, using a QP index number for identification across modules.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If QP states are stored in main memory without caching, then memory capacity is sufficient, but access speed is slow and communication efficiency deteriorates
Solution Approach 1:
The QP state cache is segmented into multiple sub-cache blocks, with each sub-cache block dedicated to a specific module. This segmentation allows each module to have direct access to its required QP state portions without contending for the same cache resources, thereby improving access speed while distributing cache management complexity across multiple independent sub-cache blocks rather than creating a single complex centralized cache management system
Solution Approach 2:
Each sub-cache block is tailored to store specific portions of QP states that are locally relevant to its associated module. This local quality optimization ensures that each module has direct access to the specific QP state data it needs, improving access speed while avoiding the need to manage entire QP states across all modules, thus reducing overall cache management complexity
2Adaptability or versatility
If QP states are cached in a centralized manner, then access is simplified, but scalability across multiple modules is limited
Solution Approach 1:
The cache is divided into multiple independent sub-cache blocks, each accessible by specific modules. This segmentation enables scalability across multiple modules as each can access its dedicated sub-cache block without interfering with others, while the standardized interface through the distributed cache logic maintains ease of operation
Solution Approach 2:
The distributed cache logic provides a universal interface that works across all modules and sub-cache blocks. This multi-functional design allows the same cache access mechanism to serve multiple modules with different requirements, enhancing scalability while maintaining operational simplicity through a unified access protocol
3Productivity
If complete QP states are stored in each module's cache, then access efficiency is maximized, but cache memory consumption increases
Solution Approach 1:
Only the necessary portions of QP states are extracted and stored in each module's dedicated sub-cache block, rather than caching complete QP states. This extraction approach maximizes processing efficiency by having direct access to required data while minimizing cache memory usage by storing only what is locally needed
Solution Approach 2:
Each module caches only the partial QP state data that is relevant to its specific functions, rather than caching complete QP states. This partial action approach achieves sufficient processing efficiency for each module's workload while significantly reducing overall cache memory consumption compared to storing full QP states in every module
Data Source
AI summary
A method for managing a distributed cache of a host channel adapter (HCA) that includes receiving a work request including a QP number, determining that a QP state identified by the QP number is not in the distributed cache, retrieving the QP state from main memory, and identifying a first portion and a second portion of the QP state. The method further includes storing the first portion into a first entry of a first sub-cache block associated with the first module, where the first entry is identified by a QP index number, storing the second portion into a second entry of a second sub-cache block associated with the second module, where the second entry is identified by the QP index number; and returning the QP index number of the QP state to the first module and the second module.