ML Accelerator Weight Loading With Parallel Decompression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increasing capacity of data loaded to machine learning accelerators results in longer loading times, necessitating improved methods to enhance data loading speed.
Innovation Solution
A machine learning accelerator with data controllers that decompress compressed weight data and monitor decompression completion, along with a computing device that uses multiple DMA masters to read and transfer data in parallel, optimizing data loading through simultaneous programming and parallel data transfer.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If compressed weight data is loaded to the machine learning accelerator, then the data loading speed is improved, but the decompression time increases
Solution Approach 1:
The accelerator is divided into multiple independent data controllers (first data controller, second data controller) and accelerator cores (first accelerator core, second accelerator core). Each data controller independently handles decompression for its corresponding accelerator core, enabling parallel decompression operations that reduce total decompression time while maintaining improved data loading speed through compression.
Solution Approach 2:
Weight data is pre-compressed before being loaded to the accelerator. The compression is performed in advance during data preparation, so that when data is transferred to the accelerator, it arrives in compressed form ready for immediate decompression by the data controllers, improving loading speed while the decompression happens in parallel during operation.
2Productivity
If multiple data controllers perform decompression in parallel, then the decompression throughput is improved, but the device complexity increases
Solution Approach 1:
The decompression function is segmented into multiple independent data controllers, each handling a portion of the total decompression workload. This segmentation enables parallel processing that increases throughput while keeping each individual controller relatively simple in structure, as each controller is a replicated instance of the same basic decompression unit.
Solution Approach 2:
Each data controller is designed as a universal, reusable component that can independently perform the complete decompression function. The first and second data controllers are identical in capability, allowing them to be replicated and configured in parallel without increasing the complexity of the basic unit, thereby scaling throughput while maintaining modular simplicity.
3Quantity of substance
If the data capacity is increased, then the machine learning model capability is improved, but the loading time increases
Solution Approach 1:
The data is transformed from uncompressed to compressed form, changing its physical state and density. This parameter change allows more data capacity to be stored and transferred in the same time frame, as compressed data occupies less transfer bandwidth and can be loaded faster despite the larger ultimate capacity required.
Solution Approach 2:
Large capacity data is divided and handled by multiple parallel data controllers that simultaneously decompress different portions. This segmentation allows the system to process large data capacities through parallel channels, preventing the loading time from increasing linearly with data capacity.
Data Source
AI summary
Disclosed is a machine learning accelerator which includes a first data controller that stores original length information indicating an original length, receives first data with a first length, and decompresses the first data with the first length to output second data with the original length, a second data controller that stores the original length information, receives third data with a second length shorter than the first length, and decompresses the third data with the second length to output fourth data with the original length, a first accelerator core that receives the second data with the original length from the first data controller and performs a first machine learning-based operation, and a second accelerator core that receives the fourth data with the original length from the second data controller and performs a second machine learning-based operation.