ML Accelerator Weight Decompression Using a Polar Encoder
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing accelerator devices struggle to efficiently support machine learning-based inference due to limitations in processing parallel operations and high power consumption.
Innovation Solution
The proposed accelerator device includes an interface circuit, a memory, a polar encoder, and an accelerator core. The polar encoder performs polar encoding on compressed weight data to decompress it, and the accelerator core loads the decompressed data to perform machine learning-based inference.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a graphics processor is used to perform machine learning-based inference, then parallel operations are supported, but the processor is not designed for machine learning and has limitations in supporting machine learning
Solution Approach 1:
The system separates the graphics processor's role (performing parallel operations) from the machine learning-specific processing. A dedicated polar encoder/decoder unit handles the compression and decompression of weight data, while the graphics processor handles the parallel computation, allowing each component to operate at its optimal capability without compromising the other.
2Quantity of substance
If weight data is stored in compressed format to reduce memory usage, then storage efficiency is improved, but decompression is required before use which increases processing time
Solution Approach 1:
The polar encoder performs compression of weight data in advance before the data is stored in memory. This preliminary action ensures that when the weight data is needed for inference, it is already in the compressed format, and the decompression can be performed efficiently by the dedicated polar decoder unit, minimizing the impact on processing time.
3Ease of operation
If weight data is decompressed before loading into the accelerator core, then the accelerator core can process the data, but power consumption increases
Solution Approach 1:
The polar decoder acts as an intermediary component between the compressed weight data in memory and the accelerator core. It performs the decompression operation in a dedicated hardware unit that is optimized for this specific task, rather than using the general-purpose accelerator core for both decompression and inference, thereby reducing the overall power consumption of the system.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Disclosed is an accelerator device which includes an interface circuit that communicates with an external device, a memory that stores first data received through the interface circuit, a polar encoder that performs polar encoding with respect to the first data provided from the memory and to output a result of the polar encoding as second data, and an accelerator core that loads the second data. The first data are compressed weight data, the second data are decompressed weight data, the accelerator core is configured to perform machine learning-based inference based on the second data, and the first data are variable in length.