Dynamic Cache and Spooler Allocation for HWA Data Throughput
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The bottleneck in data throughput for complex computational systems with hardware accelerators shifts from processing to data input, leading to reduced productivity when data input rates are suboptimal.
Innovation Solution
A method and system that dynamically allocate cache memory and spoolers based on the quantities and properties of hardware accelerators and storage units to maximize data input bandwidth, utilizing a hierarchical cache scheme and direct memory access to ensure continuous data flow without relying on common data buses or processors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is read from storage instances to hardware accelerators using conventional methods, then data processing can be performed, but the data input rate is suboptimal causing the bottleneck to shift from processing to data acquisition
Solution Approach 1:
The patent implements preliminary action by pre-allocating cache memory space and spoolers to hardware accelerators before data processing tasks begin. This allows data to be staged and prepared in advance in the cache memory hierarchy and spooler buffers, so that when processing starts, data is already positioned for optimal transfer, eliminating wait time during actual processing.
Solution Approach 2:
The patent introduces cache memory and spoolers as intermediary components between storage instances and hardware accelerators. These intermediaries buffer and manage data flow, decoupling the storage subsystem from the processing subsystem. The spoolers specifically act as mediators that regulate data transfer rates to match HWA input capabilities, preventing bottlenecks.
2Productivity
If cache memory space is increased to buffer more data, then data input continuity improves, but memory resource consumption increases
Solution Approach 1:
The patent applies local quality by allocating cache memory space specifically and locally to each hardware accelerator based on its individual data throughput requirements and processing characteristics. Rather than providing uniform memory allocation across all accelerators, the system tailors cache size and spooler capacity to each HWA's specific needs, optimizing the balance between data input continuity and memory resource consumption for each component.
Solution Approach 2:
The patent implements dynamic allocation of cache memory space and spoolers to hardware accelerators. The allocation is not static but adapts based on the specific requirements of each HWA and the current workload characteristics. This dynamic approach allows the system to optimize memory resource consumption by allocating only the necessary buffer capacity required for each accelerator's operational needs.
Data Source
AI summary
The present invention discloses a method for optimizing the throughput of hardware accelerators (HWAs) in a computerized abstraction system, by utilizing the maximal data input bandwidth to the said HWAs.The method is comprised of the following steps:dynamically obtaining the quantities and properties of HWAs and storage units within the computerized abstraction systemdynamically allocating cache memory space per each of the HWAs, according to the said obtained quantities and properties, to minimize the time required for reading data from storage instances to the said HWAdynamically allocating spoolers per each of the HWAs, according to the said obtained quantities and properties, to buffer the input data and ensure a continuous flow of input data, in the target HWA's maximal input bandwidth.