ANN Memory Controller Data Locality Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional artificial neural network (ANN) models face issues with high power consumption, heating, and performance bottlenecks due to low memory bandwidth and latency, which hinder processing performance and require an optimized memory system to improve computational efficiency.
Innovation Solution
An ANN memory system that utilizes ANN data locality information to predict and prepare data access requests, optimizing memory operations by rearranging data in a read-burst mode and controlling memory access to minimize latency and bandwidth issues.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional memory access methods are used for ANN models, then memory operations are simple and straightforward, but memory bandwidth is insufficient and latency is high causing performance bottlenecks
Solution Approach 1:
The memory controller performs preliminary actions by predicting future data access requests based on ANN data locality patterns before the processor actually needs the data. Data is pre-fetched and prepared in advance, eliminating waiting time and ensuring data is ready when needed, thus reducing memory latency without compromising processing performance.
Solution Approach 2:
The system dynamically adapts memory access patterns by learning and exploiting ANN data locality characteristics. The memory controller adjusts data fetching strategies based on observed access patterns, transforming static memory operations into dynamic, adaptive processes that optimize bandwidth utilization and reduce latency for different ANN workloads.
2Productivity
If conventional memory access methods are used for ANN models, then memory operations are simple, but power consumption is high due to frequent memory access and processor idle states
Solution Approach 1:
By performing preliminary data preparation and prediction, the system reduces the frequency of processor-stalled memory access operations. Data is staged in advance in appropriate memory locations, reducing the number of active memory transactions and processor idle cycles, thereby lowering overall power consumption while maintaining operation efficiency.
Solution Approach 2:
The memory controller maintains continuous useful action by keeping data pipelines filled through predictive pre-fetching. This eliminates processor idle states where no useful work is being done, ensuring continuous computation with minimized energy waste from frequent start-stop cycles, thus improving operation efficiency while reducing power consumption.
3Productivity
If data is not prepared in advance, then memory operations are simple and on-demand, but memory effective bandwidth is reduced and data supply is delayed
Solution Approach 1:
The memory controller performs preliminary data preparation by predicting and fetching required data before the processor needs it. This advance preparation ensures that data is ready in the memory hierarchy, maximizing effective bandwidth utilization during actual computation and enabling faster computational processing speed without data supply delays.
Data Source
AI summary
A memory system of an artificial neural network (ANN) includes a processor configured to process an ANN model; and an ANN memory controller configured to control a rearrangement of data of the ANN model stored in a memory and to operate the data of the ANN model stored in the memory in a read-burst mode based on ANN data locality information of the ANN model. The ANN memory controller may receive pre-generated ANN data locality information, or the processor may generate a plurality of data access requests sequentially so that the ANN memory controller may generate the ANN data locality information by monitoring the plurality of data access requests. The ANN memory controller prepares, based on an artificial neural network data locality, data before receiving a request from the processor in order to reduce a delay in the data supply of the memory to the processor.