Machine Learning Accelerator with Parallel Weight Decompression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increasing capacity of data loaded to machine learning accelerators results in prolonged data loading times, necessitating improved methods to enhance data loading speed.
Innovation Solution
A machine learning accelerator with data controllers configured to decompress compressed weight data and monitor decompression completion, utilizing multiple DMA masters for parallel data reading and decompression, and performing machine learning operations based on decompressed data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If weight data is compressed to decrease loading time, then data loading speed is improved, but data decompression complexity increases
Solution Approach 1:
The decompression process is divided into multiple parallel data controllers (first data controller, second data controller, etc.), each handling a portion of the decompression task. This segmentation allows simultaneous decompression of different data portions, reducing overall decompression time while distributing the complexity across multiple dedicated units rather than one complex sequential processor
Solution Approach 2:
Original length information is stored in advance before compression occurs. This pre-stored metadata enables each data controller to immediately know the target length for decompression without requiring complex runtime analysis or iteration, simplifying the decompression logic while maintaining accuracy
2Productivity
If multiple data controllers perform parallel decompression, then data loading speed is improved, but device complexity increases
Solution Approach 1:
Multiple data controllers are designed with identical or highly similar structures, each capable of independent decompression operations. This homogeneity simplifies the overall system architecture by using replicated simple units rather than one complex controller, making the system more manageable and scalable while achieving parallel processing goals
Solution Approach 2:
Each data controller is designed as a universal decompression unit that can handle different portions of compressed weight data through parallel operation. The controllers perform the same decompression function simultaneously on different data segments, providing multi-functionality at the system level while keeping individual controller complexity low
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Disclosed is a machine learning accelerator which includes a first data controller that stores original length information indicating an original length, receives first data with a first length, and decompresses the first data with the first length to output second data with the original length, a second data controller that stores the original length information, receives third data with a second length shorter than the first length, and decompresses the third data with the second length to output fourth data with the original length, a first accelerator core that receives the second data with the original length from the first data controller and performs a first machine learning-based operation, and a second accelerator core that receives the fourth data with the original length from the second data controller and performs a second machine learning-based operation.