Heterogeneous Memory Stack for Deep Learning Accelerator Energy Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing integrated circuit devices for Artificial Neural Networks (ANNs) face challenges in reducing energy consumption and computation time, particularly due to inefficient memory access patterns and lack of specialized hardware for parallel vector and matrix operations.
Innovation Solution
The integration of a Deep Learning Accelerator (DLA) with specialized programmable hardware and a heterogeneous memory system architecture, optimized for parallel vector and matrix calculations, and coupled with a high-bandwidth connection to random access memory, enables efficient computation and reduced energy consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a Deep Learning Accelerator with specialized hardware for parallel vector and matrix operations is integrated, then computation speed and productivity are improved, but device complexity increases
Solution Approach 1:
The memory system is segmented into different types (volatile memory and non-volatile memory) with distinct functions. The Deep Learning Accelerator is separated from the memory subsystem, allowing independent optimization of each component while maintaining high-speed data access through dedicated memory interfaces.
Solution Approach 2:
A memory controller acts as an intermediary between the Deep Learning Accelerator and the memory subsystem, managing data transfers and optimizing memory access patterns. This intermediary component simplifies the interaction between the accelerator and memory, reducing the overall system complexity while maintaining high productivity.
2Use of energy by moving object
If a heterogeneous memory system architecture is used to optimize memory access patterns, then energy consumption is reduced, but device complexity increases
Solution Approach 1:
Different memory types are assigned to different locations in the memory hierarchy based on their specific characteristics. Non-volatile memory is used for storing model data and weights that require persistence, while volatile memory is used for temporary computation data. This local quality assignment optimizes energy consumption by using memory with appropriate retention characteristics for each data type.
Solution Approach 2:
The system performs preliminary actions by loading frequently accessed data into faster volatile memory from non-volatile memory before computation begins. This prefetching mechanism reduces the need for frequent slow memory accesses during computation, thereby reducing overall energy consumption without requiring complex real-time memory management.
3Speed
If high-bandwidth connection to random access memory is implemented, then data access speed is improved, but manufacturing complexity increases
Solution Approach 1:
The memory interface is designed with multiple parallel data channels, transitioning from a single-dimensional data bus to a multi-dimensional parallel architecture. This dimensional expansion allows high-bandwidth data transfer by simultaneously accessing multiple memory locations through parallel channels, achieving high data access speed without requiring excessively complex single-channel designs.
Data Source
AI summary
Systems, devices, and methods related to a Deep Learning Accelerator and memory are described. For example, an integrated circuit (IC) device includes a first stack of IC dies connected to a plurality of second stacks of IC dies. The first stack has a first die of a memory controller and processing units of the Deep Learning Accelerator and at least one second die that is stacked on the first die to provide a first type of memory. Each of the second stacks has a base die and at least a third die and a fourth die having different types of memory. The base die has logic circuit configured to copy data within the same stack in response to commands from the memory controller and has a second type of memory usable as die cross buffer.