NAND Flash DNN Accelerator for Memory Capacity and Bandwidth Trade-off
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing high-bandwidth memory (HBM) DRAM-based systems lack the capacity to efficiently process large-scale deep neural network (DNN) models, particularly for natural language processing, due to their limited memory capacity and high computational complexity, leading to inefficiencies in training large DNN models.
Innovation Solution
A DNN accelerator system utilizing a NAND flash-based memory system with a controller and processor architecture that enables data parallel training, incorporating a tensor buffer and compute core for efficient matrix operations, and a hardware-based flash memory system controller to manage data access patterns, thereby improving bandwidth and durability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If HBM DRAM-based memory system is used, then high bandwidth is achieved, but memory capacity is insufficient for large-scale DNN models
Solution Approach 1:
The system divides the memory system into two segments: HBM DRAM for high-speed data access during computation and NAND flash memory for large-capacity model parameter storage. This segmentation allows each component to perform its strength function, resolving the contradiction between bandwidth and capacity.
Solution Approach 2:
A flash memory system controller acts as an intermediary between the HBM DRAM and NAND flash memory, managing data transfer and access patterns. This intermediary enables efficient utilization of both high-bandwidth memory and large-capacity storage in the same system.
2Quantity of substance
If NAND flash memory is used, then large model capacity is achieved, but wear-leveling issues reduce durability
Solution Approach 1:
The system performs preliminary actions by pre-processing training data and organizing model parameters before writing to NAND flash memory. This includes data formatting, compression, and optimization of access patterns to minimize write operations and extend memory lifespan.
Solution Approach 2:
The system changes operational parameters of the NAND flash memory by optimizing read/write voltages, access timing, and data organization formats. These parameter adjustments reduce stress on the memory cells and improve overall durability during intensive DNN training operations.
3Adaptability or versatility
If DNN model size is increased, then model capability is improved, but computational complexity increases
Solution Approach 1:
The system segments the computational workload across multiple accelerator nodes, each handling specific layers or operations of the large DNN model. This segmentation distributes computational complexity while maintaining overall model capability through coordinated parallel processing.
Solution Approach 2:
The system transitions from single-node sequential processing to multi-node parallel processing, adding a spatial dimension to computation. This dimensional change allows the system to handle increased model complexity by distributing work across multiple processing units simultaneously.
4Speed
If existing HBM DRAM-based system is used, then high bandwidth is achieved, but efficiency for large DNN models deteriorates
Solution Approach 1:
The system segments the memory hierarchy into HBM DRAM for active computation data and NAND flash for model parameters, with a controller managing data flow between them. This segmentation enables the system to maintain high bandwidth for training operations while efficiently utilizing large-capacity storage, thereby improving overall training efficiency for large DNN models.
Data Source
AI summary
A DNN accelerator system includes a plurality of accelerator nodes each including a plurality of NAND flash memories, a flash memory system (FMS) controller for controlling the plurality of NAND flash memories, and a tensor buffer, and a processor configured to generate an operation sequence of the plurality of accelerator nodes, in which a DNN model is trained in a data parallel manner using the plurality of accelerator nodes.