Workload Scheduler With Queue-Based CPU-Offloaded Memory Allocation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Memory allocation and management in cloud computing systems is resource-intensive and contributes to latency and jitter, with existing software implementations like TCMalloc, glibc's malloc, and jemalloc requiring significant CPU cycles and central table access.
Innovation Solution
A work scheduler manages memory allocation and deallocation by offloading these tasks to a queue management device (QMD) that uses a volatile memory to allocate logical queues, reducing the need for CPU involvement and optimizing core-to-core communication through a system of queues and credit pools.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If software-based memory allocation (TCMalloc, glibc's malloc, jemalloc) is used, then memory management functionality is provided, but CPU utilization increases and latency/jitter occurs
Solution Approach 1:
The patent introduces a queue management device (QMD) as an intermediary component between CPUs and memory allocation requests. The QMD maintains a pool of memory segments and uses work queues to manage allocation requests, acting as a mediator that handles memory management tasks without requiring direct CPU intervention for each allocation operation.
Solution Approach 2:
The system enables self-service memory allocation where the QMD automatically manages memory segment allocation and deallocation without CPU involvement. The credit-based mechanism allows cores to autonomously request and receive memory segments from the QMD, which independently manages its internal memory pools and queue operations.
2Reliability
If software-based memory allocation is used, then memory management is performed, but latency and jitter increase
Solution Approach 1:
The QMD pre-allocates and maintains pools of memory segments in advance, so that when allocation requests arrive, memory can be immediately granted without requiring time-consuming search or allocation operations. The work queues are also pre-configured to handle requests efficiently.
Solution Approach 2:
The patent replaces software-based memory management with a hardware-based QMD that uses dedicated circuits and logic for queue management and memory allocation. This hardware implementation eliminates the CPU cycle overhead and timing variability inherent in software implementations.
3Ease of operation
If CPUs directly manage memory allocation, then control is maintained, but CPU cycles are consumed for management tasks
Solution Approach 1:
The patent extracts memory management functionality from the CPU by implementing it in a separate QMD component. The QMD independently handles all memory allocation, deallocation, and queue management operations, freeing the CPU from these management tasks while maintaining system control through the credit-based authorization mechanism.
Data Source
AI summary
Examples described herein relate to a work scheduler that includes at least one processor and at least one queue. In some examples, the work scheduler receives a request to allocate a region of memory and based on availability of a memory segment associated with a central cache to satisfy the request to allocate a region of memory, provide a memory allocation using an available memory segment entry associated with the central cache from the at least one queue. In some examples, the work scheduler assigns a workload to a processor and controls when to pre-fetch content relevant to the workload to store in a cache or memory accessible to the processor based on a position of the workload in a work queue associated with the processor.