Next Read Index Caching for Low-Power Memory Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Convolutional neural networks (CNNs) face increased power consumption due to frequent access of weight data from tightly coupled memory (TCM), disrupting operations and consuming excessive bandwidth.
Innovation Solution
Implementing a data cache and a next read index storage element to store indices of data, allowing cached data to be reused, reducing the need for direct memory access and minimizing power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If weight data is frequently accessed from tightly coupled memory (TCM), then neural network operations can be performed, but power consumption increases and bandwidth is exhausted
Solution Approach 1:
The patent loads weight data into the data cache in advance before it is needed for neural network operations. The next read index storage element stores indices of weight data that will be needed soon, allowing the system to prefetch and cache data proactively. This preliminary action reduces the frequency of TCM accesses during actual computation, thereby lowering power consumption while maintaining operational continuity.
Solution Approach 2:
The patent introduces a data cache as an intermediary between the TCM and the neural network processing units. This cache acts as a buffer that stores frequently accessed weight data, reducing the number of direct TCM accesses. The next read index storage element serves as a mediator that tracks which cached data will be needed next, enabling efficient data reuse without continuous memory access.
2Productivity
If weight data is accessed from TCM, then neural network computation can proceed, but operations are disrupted and bandwidth is consumed
Solution Approach 1:
The system proactively loads weight data into the data cache before it is needed for computation. By using the next read index storage element to track upcoming data needs, the system can prepare data in advance, ensuring that when computation requires weight data, it is already available in the cache. This eliminates disruptions caused by TCM access delays and maintains continuous operational flow.
Solution Approach 2:
The data cache serves as an intermediary buffer that decouples the neural network processing units from direct TCM dependencies. Weight data is first loaded into this intermediate cache structure, allowing computations to proceed without being blocked by TCM access timing. This intermediary layer ensures operation continuity by providing locally available data for sustained computation.
3Use of energy by moving object
If a data cache is implemented to reduce TCM access, then power consumption decreases, but device complexity increases
Solution Approach 1:
The patent uses a simple next read index storage element to track which weight data will be needed next, enabling proactive caching without complex prediction algorithms. This straightforward index-based approach allows the system to load data into the cache in advance based on minimal bookkeeping, reducing power consumption while adding only slight structural complexity rather than requiring sophisticated data prediction mechanisms.
Solution Approach 2:
The data cache is implemented as a simple intermediary storage structure between TCM and processing units. Rather than replacing the entire memory hierarchy with complex structures, the patent adds a relatively simple cache layer with an index storage element that tracks next read requirements. This intermediary approach reduces power consumption through localized data storage while introducing manageable complexity rather than fundamental architectural overhaul.
Data Source
AI summary
Certain aspects of the present disclosure generally relate to electronic circuits and, more particularly, to techniques for memory access. Certain aspects provide a method for memory access. The method generally includes identifying whether data to be accessed is stored in a data cache coupled to a memory, storing an index associated with a line in the data cache in a next read index storage element based on the identification, and processing the data from the data cache based on the index