Large language model accelerator architecture based on three-dimensional NAND flash memory
By using an accelerator architecture based on 3D NAND flash memory and a dynamic scheduling method, the storage and computation bottlenecks of large language models are solved, efficient matrix-vector multiplication and addition operations are achieved, inference performance and energy efficiency are improved, and hardware costs are reduced.
Patent Information
- Application Number
- CN202511558183.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-10-29
- Publication Date
- 2026-02-10
- Estimated Expiration
- 2045-10-29
AI Technical Summary
There are bottlenecks in the computation and storage of large language models in existing technologies. In particular, in DRAM-based architectures, it is difficult to balance storage capacity and access bandwidth requirements. Weight read latency is high, data migration overhead is large, and the lack of dynamic scheduling mechanism leads to resource waste.
The design incorporates a large language model accelerator architecture based on 3D NAND flash memory, including DRAM, auxiliary computing units, NAND array chips, flash logic chips, and interconnect channels. By performing matrix-vector multiplication and addition operations within the flash memory and combining dynamic scheduling methods, the attention mechanism and feedforward neural network computation are optimized.
It improves the inference performance and energy efficiency of large language models, reduces weight transfer overhead, alleviates the memory wall bottleneck, supports pipelined access and dynamic scheduling, and reduces hardware costs.