NPU-DRAM Memory Control for Predictive ANN Data Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional artificial neural network models face issues such as high-power consumption, heating, and bottlenecks due to low memory bandwidth and latency, leading to degraded operation performance and inefficient data processing.
Innovation Solution
An artificial neural network memory system that utilizes artificial neural network data locality to optimize processor operations by predicting and preparing data access requests in advance, minimizing memory latency and bandwidth issues through an artificial neural network memory controller.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional memory operations are used for artificial neural network processing, then the system can operate with standard memory bandwidth and latency, but the processing performance is degraded and power consumption increases due to bottlenecks
Solution Approach 1:
The memory controller predicts future data access requests by analyzing access patterns and prepares data in advance before the processor actually needs it. This preliminary action eliminates waiting time and ensures data is ready when needed, improving processing performance while reducing the need for high-speed memory operations that consume excessive power.
Solution Approach 2:
The memory controller autonomously monitors its own operation and generates predictions about future data needs without external intervention. This self-service capability allows the system to optimize memory operations dynamically, improving performance while maintaining energy efficiency through intelligent, autonomous memory management.
2Speed
If data is not prepared in advance in memory, then memory operations can proceed with standard bandwidth usage, but data supply delay occurs and processor operation is bottlenecked
Solution Approach 1:
The memory controller performs preliminary data preparation by predicting which data will be needed next and loading it into the memory buffer in advance. This eliminates the latency that would otherwise occur when data needs to be fetched from slower storage, ensuring high-speed data supply to the processor without requiring continuously high memory bandwidth.
3Productivity
If memory bandwidth is increased to handle large artificial neural network models, then data processing speed improves, but power consumption and heating increase
Solution Approach 1:
By preparing data in advance using prediction algorithms, the system can use lower memory bandwidth when actually transferring data to the processor, since the data is already staged and ready. This reduces the energy consumption associated with high-speed memory operations while maintaining high processing throughput.
Solution Approach 2:
The memory controller's autonomous prediction and data preparation capabilities allow the system to optimize memory bandwidth usage dynamically, using high bandwidth only when necessary and lower bandwidth during normal operation, thereby reducing overall energy consumption while maintaining processing speed.
Data Source
AI summary
An electronic device is disclosed comprising a dynamic random-access memory (DRAM) storing at least part of data of an artificial neural network (ANN) model, a neural processing unit (NPU), and a memory controller. The NPU processes inference of the ANN model according to input data and outputs an inference result. The NPU generates a data access request based on a predetermined sequence of operations, including read and write operations for the ANN model, wherein the sequence is determined at compile time for the ANN model. The memory controller, electrically connected to the NPU and the DRAM, receives the data access request from the NPU and controls the DRAM accordingly.