Memory Offloading via Unit Segmentation for Accelerator Capacity Limits
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In computing environments, the limited memory capacity of accelerators and system memory restricts the size of problems that can be processed, especially for applications with sizes that are powers of 2, such as quantum computing simulations, due to the occupation of system memory by OS kernels and drivers, leaving only half the memory available for problem storage.
Innovation Solution
A method is introduced to offload problems of size 2n from processing circuitry to accelerators by dividing the problem into units of size 2u, allocating them to accelerators, and using temporary buffers for inter-unit calculations, allowing efficient use of both system and device memory.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If all data is stored on system memory and only copied to accelerators for calculation, then the problem size is limited by accelerator memory capacity, but system memory utilization is wasted
Solution Approach 1:
The problem data is divided into multiple units that are distributed across both system memory and accelerator memory. Each unit can be independently accessed and processed, allowing the system to overcome the memory capacity limitations of individual accelerators while maintaining efficient access patterns through the segmentation of data into manageable chunks.
Solution Approach 2:
The patent introduces a hybrid memory architecture that adds another dimension to data storage by utilizing both system memory and accelerator memory simultaneously. This multi-dimensional memory space allows problems larger than any single memory device to be solved, as data can be distributed across the combined memory resources of the system.
2Quantity of substance
If problem data is divided and allocated across system memory and device memory, then maximum problem size is doubled, but memory management complexity increases
Solution Approach 1:
The patent employs an intermediary memory management mechanism that coordinates data distribution between system memory and device memory. This intermediary layer handles the complexity of allocating data units across different memory spaces, providing a simplified interface for applications while managing the underlying complexity of cross-memory operations and data coherence.
3Speed
If accelerators are used for high performance calculation, then calculation speed is improved, but available memory space on accelerators is limited
Solution Approach 1:
The patent merges the memory spaces of the system memory and accelerator memory into a unified addressable space. This combination allows the system to leverage the high calculation speed of accelerators while accessing data from the larger system memory, effectively overcoming the memory capacity limitations of accelerators without sacrificing computational performance.
Data Source
AI summary
A computer-implemented method for offloading a problem having 2n size from processing circuitry to one or more accelerators is disclosed. The processing circuitry and the one or more accelerators include respective memories. In the method, a problem having 2n size is divided into a plurality of units each having 2u size. At least a part of the units is allocated to the one or more accelerators. A determination is made as to whether there is a remaining part of the units to be allocated onto the processing circuitry. A temporary buffer is prepared on each memory of at least the one or more accelerators. The temporary buffer is used for storing a copy of a dependent unit stored on a different memory, during inter-unit calculation.