Memory Traffic Prioritization for Multi-Process Workloads
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Microservices within an application can have varying memory usage characteristics, leading to inefficiencies in memory management and increased latency when accessing remote fabric-attached memory (FAM) resources.
Innovation Solution
A system and method for prioritizing memory traffic in a memory-semantic fabric by identifying source processes for memory requests and allocating cycles using a weighted round-robin scheme based on memory priorities, which are determined by observing memory request rates or wait times.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If memory requests from multiple microservices are handled with equal priority, then fairness is maintained, but latency increases for memory-intensive applications
Solution Approach 1:
The patent applies local quality by assigning different priority levels to different microservices based on their individual memory usage characteristics. The controller identifies source processes and allocates memory bandwidth differently to each process, allowing memory-intensive processes to receive higher priority while less intensive processes receive standard priority, thus resolving the contradiction between fairness and latency.
2Device complexity
If memory bandwidth is allocated uniformly across all processes, then resource distribution is simple, but performance of memory-intensive applications deteriorates
Solution Approach 1:
The patent implements dynamic memory bandwidth allocation where the controller continuously monitors memory usage characteristics of different microservices and adjusts priority levels accordingly. This dynamic approach allows the system to optimize performance for memory-intensive applications when they are active while maintaining simple uniform allocation when such applications are not present, resolving the contradiction between allocation simplicity and application performance.
3Productivity
If granular prioritization of memory requests is implemented, then performance of memory-intensive applications improves, but system complexity increases
Solution Approach 1:
The patent introduces a controller as an intermediary component that manages memory traffic prioritization. This controller sits between the microservices and the fabric-attached memory, intelligently identifying source processes and allocating bandwidth based on their needs. By centralizing the complexity in this intermediary component, the system achieves granular prioritization and improved performance while keeping the complexity localized rather than distributed throughout the entire system.
Data Source
AI summary
The disclosed device includes a memory-semantic fabric comprising memory components accessible by multiple processors and a controller for the memory-semantic fabric. The controller receives, from multiple processes, memory requests for a memory-semantic fabric. The controller also identifies, within the processes, a source process for each of the memory requests and prioritizes forwarding the memory requests to the memory-semantic fabric based on the source processes. Various other methods, systems, and computer-readable media are also disclosed.