Cache Allocation for RDMA Connection Contexts
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In microservices environments, the limited cache capacity for RDMA connection states leads to increased latency and potential violations of quality of service (QoS) and service-level-agreement (SLA) parameters due to cache misses, especially as workloads become more complex and the number of active network connections grows.
Innovation Solution
Implementing a cache manager that dynamically allocates and evicts cache space based on Resource Manager Identifiers (RMIDs) assigned to connections, prioritizing reliable and long-lived connections while de-prioritizing short-lived and unreliable ones, to optimize cache utilization and reduce latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If cache size is increased to store more connection states, then cache hit rate is improved, but device complexity and cost increase
Solution Approach 1:
The patent applies local quality by differentiating cache allocation based on connection characteristics. Different cache line sizes (e.g., 64-byte vs. 128-byte) are allocated to different types of connections (short-lived vs. long-lived), ensuring that each connection type receives appropriate cache resources without uniformly increasing overall cache size. This resolves the contradiction by improving cache hit rate for specific connection types without proportionally increasing device complexity.
2Device complexity
If cache size is kept limited to maintain simplicity, then device complexity is reduced, but latency increases due to cache misses
Solution Approach 1:
The patent implements dynamic cache allocation where cache resources are not statically assigned but dynamically adjusted based on connection characteristics. The system dynamically determines whether to allocate 64-byte or 128-byte cache lines based on connection type, and dynamically manages cache eviction priorities. This dynamic approach allows the limited cache to adaptively optimize for current workload patterns, reducing latency without requiring a larger static cache.
3Ease of operation
If uniform cache allocation is used for all connections, then ease of operation is improved, but productivity decreases due to inefficient cache utilization
Solution Approach 1:
The patent changes the parameter of cache line size based on connection characteristics. Instead of using a uniform cache allocation, the system varies cache line sizes (64-byte for short-lived connections, 128-byte for long-lived connections) to optimize cache utilization. This parameter change improves productivity by ensuring that cache resources are appropriately sized for each connection type, reducing cache misses and improving packet processing throughput while maintaining manageable complexity through automated classification.
Data Source
AI summary
Examples described herein relate to a network interface device comprising: a host interface, a direct memory access (DMA) engine, and circuitry to allocate a region in a cache to store a context of a connection. In some examples, the circuitry is to allocate a region in a cache to store a context of a connection based on connection reliability and wherein connection reliability comprises use of a reliable transport protocol or non-use of a reliable transport protocol. In some examples, the circuitry is to allocate a region in a cache to store a context of a connection based on expected length of runtime of the connection and the expected length of runtime of the connection is based on a historic average amount of time the context for the connection was stored in the cache. In some examples, the circuitry is to allocate a region in a cache to store a context of a connection based on content transmitted and the content transmitted comprises congestion messaging payload or acknowledgement. In some examples, the circuitry is to allocate a region in a cache to store a context of a connection based on application-specified priority level and the application-specified priority level comprises an application-specified traffic class level or class of service level.