CXL Memory Expander for Metadata-Based Accelerator Data Allocation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In heterogeneous computing environments, the management of data allocation by a central processing unit (CPU) to accelerators leads to reduced performance due to frequent interrupts and repetitive address transmissions, which hampers efficient data processing.
Innovation Solution
A memory expander with a controller that manages data allocation based on metadata from the CPU, operating in a management mode to facilitate seamless data transfer and storage between the CPU, accelerators, and memory devices using a Compute Express Link (CXL) interface.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the CPU manages data allocation to accelerators directly, then data transfer control is precise, but CPU utilization decreases due to frequent interrupts and repetitive address transmissions
Solution Approach 1:
The patent introduces a memory expander as an intermediary device between the CPU and accelerators. The memory expander includes a controller that receives metadata and management requests from the CPU, then autonomously manages data allocation to multiple accelerators. This intermediary handles repetitive tasks such as address transmissions and data routing, freeing the CPU from frequent interrupts while maintaining precise control over data transfer operations.
Solution Approach 2:
The patent segments the data management functionality by separating control functions. The CPU retains high-level management responsibilities while delegating specific data allocation tasks to the memory expander's controller. This segmentation allows the CPU to focus on critical decision-making while the memory expander handles routine data distribution, thereby improving CPU utilization without sacrificing control precision.
2Measurement precision
If the CPU handles all data transmission to accelerators, then data allocation accuracy is maintained, but data processing speed decreases due to continuous CPU intervention
Solution Approach 1:
The patent implements preliminary action by having the CPU send metadata and management requests to the memory expander in advance. The memory expander controller then autonomously executes data allocation based on pre-provided information, eliminating the need for continuous CPU intervention during actual data transmission. This preliminary setup maintains allocation accuracy while enabling faster data processing speeds.
Solution Approach 2:
The memory expander controller performs self-service by autonomously managing data allocation to accelerators without requiring continuous CPU involvement. Once initialized with metadata and management requests, the controller independently handles address transmissions, data routing, and allocation decisions, thereby maintaining accuracy while significantly improving data processing speed.
3Productivity
If the system uses a memory expander for data management, then CPU utilization improves, but system complexity increases
Solution Approach 1:
The memory expander is designed with multi-functionality, serving as both a memory expansion device and a data management controller for multiple accelerators. By combining these functions in a single device, the patent avoids the need for separate control mechanisms, thereby improving CPU utilization without proportionally increasing system complexity. The universal design allows one device to perform multiple roles that would otherwise require additional components.
Data Source
AI summary
A memory expander includes a memory device that stores a plurality of task data. A controller controls the memory device. The controller receives metadata and a management request from an external central processing unit (CPU) through a compute express link (CXL) interface and operates in a management mode in response to the management request. In the management mode, the controller receives a read request and a first address from an accelerator through the CXL interface and transmits one of the plurality of task data to the accelerator based on the metadata in response to the read request.