ML Accelerator Weight Decoding for Faster Low-Power Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing accelerator devices struggle to efficiently support machine learning-based inference due to limitations in processing parallel operations and high power consumption.
Innovation Solution
The proposed accelerator device includes a polar encoder that performs polar encoding on compressed weight data, transforming it into decompressed weight data, which is then loaded into an accelerator core for machine learning-based inference. This design improves operating speed and reduces power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a graphics processor is used to perform machine learning-based inference, then parallel operations are supported, but the processor is not optimized for machine learning resulting in limited performance and high power consumption
Solution Approach 1:
The accelerator device is divided into specialized components: a polar encoding unit for data transformation, a computing unit for parallel operations, and a weight management system. This segmentation allows each component to be optimized for its specific function, improving overall machine learning inference performance while reducing power consumption compared to using a general-purpose graphics processor.
2Quantity of substance
If compressed weight data is stored in memory, then storage efficiency is improved, but decompression is required before use increasing processing time
Solution Approach 1:
Weight data is pre-compressed and stored in memory in a compact format. The polar encoding unit performs decompression on-demand before the computing unit processes the weight data. This preliminary compression maintains storage efficiency while the just-in-time decompression minimizes processing delays.
Solution Approach 2:
The polar encoding unit acts as an intermediary component between compressed weight data storage and the computing unit. It transforms compressed weight data into a format suitable for processing, enabling efficient storage while providing timely decompression when needed for machine learning inference operations.
3Device complexity
If fixed-length data processing is used, then processing simplicity is maintained, but variable-length weight data cannot be efficiently handled
Solution Approach 1:
The polar encoding unit dynamically adapts to variable-length weight data by performing polar encoding that accommodates different data lengths. This dynamic processing approach maintains operational simplicity while providing the flexibility needed to handle weight data of varying lengths efficiently.
Solution Approach 2:
The system changes the processing parameters through polar encoding, transforming variable-length compressed weight data into a standardized format suitable for the computing unit. This parameter transformation maintains processing simplicity while enabling efficient handling of variable-length weight data through mathematical transformation.
Data Source
AI summary
Disclosed is an accelerator device which includes an interface circuit that communicates with an external device, a memory that stores first data received through the interface circuit, a polar encoder that performs polar encoding with respect to the first data provided from the memory and to output a result of the polar encoding as second data, and an accelerator core that loads the second data. The first data are compressed weight data, the second data are decompressed weight data, the accelerator core is configured to perform machine learning-based inference based on the second data, and the first data are variable in length.