ML Accelerator Data Loading for Sparse Parameter Reuse
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning accelerators face inefficiencies in data loading due to the energy-intensive process of transferring sparse data, where zero-valued activations and weights contribute to energy consumption without aiding computation, and existing techniques struggle to maintain compute unit utilization.
Innovation Solution
Implementing a data controller with a compressed local data re-user and zero compute load skipper to selectively load only useful data sections, leveraging sparsity bitmaps to determine which data is nonzero and thus useful for computation, reducing unnecessary data transfer and energy consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If all compressed machine learning parameter data is loaded into the processor engine, then the compute unit utilization is maintained, but the energy consumption increases due to transferring zero-valued data
Solution Approach 1:
The patent extracts and removes zero-valued data sections from the compressed machine learning parameter data before loading into the processor engine. By identifying and eliminating these non-contributing data elements, the system reduces energy consumption during data transfer while preserving all computationally useful information, thus resolving the contradiction between maintaining compute unit utilization and reducing energy consumption.
Solution Approach 2:
The patent applies different handling strategies to different sections of the compressed data based on their computational value. Zero-valued sections are removed entirely, while non-zero sections are preserved and loaded. This localized differentiation allows the system to optimize energy consumption for specific data portions without compromising overall compute unit utilization.
2Use of energy by moving object
If only useful data sections are loaded into the processor engine, then the energy consumption is reduced, but the compute unit utilization may decrease
Solution Approach 1:
The patent employs a feedback mechanism where the system continuously monitors and identifies zero-valued data sections, dynamically adjusts the data loading strategy by removing these sections, and validates that compute unit utilization remains adequate. This closed-loop approach ensures that energy consumption is minimized through selective data loading while maintaining sufficient compute unit utilization by preserving all non-zero data sections.
3Reliability
If compressed machine learning parameter data is loaded every round, then the data availability is ensured, but the memory access increases unnecessarily
Solution Approach 1:
The patent extracts zero-valued data sections from the compressed machine learning parameter data before each loading operation. By removing these redundant elements, the system reduces memory access requirements while ensuring that all necessary non-zero data sections are still loaded and available for computation, thus resolving the contradiction between data availability and memory access reduction.
Solution Approach 2:
The patent discards zero-valued data sections that would otherwise be loaded into memory, as these sections provide no computational value. This selective discarding reduces unnecessary memory access operations while preserving all useful data sections, thereby reducing energy loss from memory access while maintaining data availability for computation.
Data Source
AI summary
Methods, apparatus, systems, and articles of manufacture to load data into an accelerator are disclosed. An example apparatus includes data provider circuitry to load a first section and an additional amount of compressed machine learning parameter data into a processor engine. Processor engine circuitry executes a machine learning operation using the first section of compressed machine learning parameter data. A compressed local data re-user circuitry determines if a second section is present in the additional amount of compressed machine learning parameter data. The processor engine circuitry executes a machine learning operation using the second section when the second section is present in the additional amount of compressed machine learning parameter data.