ML Accelerator Data Loading with Compressed Zero-Skip Reuse
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning accelerators face challenges in efficiently loading data due to the sparsity of weights and activations in deep neural networks, leading to energy inefficiencies and computational bottlenecks.
Innovation Solution
The implementation of a data controller with a compressed local data re-user and a zero compute load skipper, which selectively loads only useful compressed machine learning parameter data, reducing unnecessary data loading and energy consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If all machine learning parameter data is loaded into the processor engine, then the processor engine has sufficient data to execute machine learning operations, but memory access and energy consumption increase due to loading zero-valued data
Solution Approach 1:
The patent extracts only the non-zero elements from the machine learning parameter data (weights and activations) and loads them into the processor engine, while filtering out zero-valued data. This is achieved through sparsity indication information that identifies which elements are non-zero, allowing the system to load only the necessary data portions and avoid wasting energy on zero-valued data transmission and storage.
2Productivity
If compressed machine learning parameter data is loaded into the processor engine every round, then the processor engine can execute machine learning operations, but clock cycles increase due to redundant data loading
Solution Approach 1:
The patent performs preliminary processing of the machine learning parameter data by compressing it and generating sparsity indication information before loading into the processor engine. This preliminary compression and sparsity analysis allows the system to quickly identify and load only the necessary non-zero elements during execution rounds, avoiding redundant data loading operations and reducing clock cycle consumption.
Solution Approach 2:
The patent enables the processor engine to skip loading operations for data sections that are identified as containing only zero-valued elements through the sparsity indication information. When the sparsity indication shows that a particular data section contains no non-zero elements, the system can skip the loading process for that section entirely, rushing through the necessary data transfer operations and reducing overall execution time.
3Duration of action of stationary object
If compressed machine learning parameter data is stored in local memory, then data can be retained for multiple rounds of computation, but memory access increases when loading data that is already present
Solution Approach 1:
The patent implements a feedback mechanism where the data controller checks the sparsity indication information and the current round's requirements before loading data from local memory to the processor engine. This feedback loop allows the system to determine whether data needs to be reloaded or can be skipped, based on whether the data contains non-zero elements that are necessary for the current computation round, thereby avoiding redundant memory access operations.
Data Source
AI summary
Methods, apparatus, systems, and articles of manufacture to load data into an accelerator are disclosed. An example apparatus includes data provider circuitry to load a first section and an additional amount of compressed machine learning parameter data into a processor engine. Processor engine circuitry executes a machine learning operation using the first section of compressed machine learning parameter data. A compressed local data re-user circuitry determines if a second section is present in the additional amount of compressed machine learning parameter data. The processor engine circuitry executes a machine learning operation using the second section when the second section is present in the additional amount of compressed machine learning parameter data.