Multi-chiplet DNN Accelerator Architecture for Latency Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
High performance computing (HPC) devices face significant data latency issues due to accessing weights stored in DRAM, which hampers efficient processing in machine learning and deep learning applications.
Innovation Solution
The approach involves categorizing classes into multiple categories and training separate models for each category, with each model stored in a static random-access memory (SRAM) on accelerators, allowing for parallel and independent computation, reducing model complexity and latency by minimizing DRAM access.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If weights are stored in DRAM for HPC devices, then memory capacity is sufficient, but data latency increases significantly
Solution Approach 1:
The patent divides the neural network model into multiple chiplets, each containing a portion of the weights and computation logic. This segmentation allows each chiplet to process data independently with local weight storage in SRAM, reducing the need for large external DRAM and minimizing data latency while maintaining sufficient memory capacity through distributed architecture.
2Power
If a single large model is used, then computational power is sufficient, but model complexity and latency increase
Solution Approach 1:
The patent segments the neural network into multiple smaller models distributed across different chiplets. Each chiplet contains a simplified model that can operate independently, reducing individual model complexity while collectively providing sufficient computational power through parallel processing. This segmentation also enables independent computation and reduces overall latency.
Solution Approach 2:
The patent transitions from a single large centralized model to a distributed architecture where multiple smaller models operate in parallel across different spatial dimensions (separate chiplets). This dimensional change allows the system to maintain high computational power through parallelism while reducing individual model complexity and latency.
3Quantity of substance
If weights are accessed from DRAM, then memory bandwidth is sufficient, but computation speed decreases due to latency
Solution Approach 1:
The patent introduces SRAM as an intermediary storage layer between the computation logic and external DRAM. Each chiplet includes local SRAM storage for its weights, acting as a buffer that reduces direct access to external DRAM. This intermediary approach maintains sufficient memory bandwidth through local SRAM access while significantly improving computation speed by minimizing latency-prone external memory accesses.
Data Source
AI summary
A design method, an operating method and an electronic system are provided. The method comprises receiving a training dataset having a plurality of training data, wherein each training data is labeled to one of a plurality of classes; selecting at least one first class from the plurality of classes and establishing a first category having the at least one selected first class; training a first model with the training dataset, and using the at least one first class within the first category for verification; and implementing the first model on the accelerator.