Hierarchical Memory for CNN Data Reuse
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current convolutional neural networks face inefficiencies in memory access and reuse due to repeated data rotation near the multiply accumulate layer, leading to suboptimal memory usage and access times.
Innovation Solution
Implementing a hierarchical memory structure by partitioning memory into sections, stripes, and tiles, and utilizing a multi-level memory hierarchy that stores data components linearly in different address directions to minimize random access memory usage and maximize data reuse.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is repeatedly addressed and stored near the multiply accumulate layer, then data can be reused, but memory access latency increases and memory usage becomes inefficient
Solution Approach 1:
The patent segments the memory hierarchy into multiple levels (L1 cache, L2 cache, main memory) with different access characteristics. Each level is optimized for specific access patterns, allowing frequent data reuse at lower levels while maintaining overall system performance.
Solution Approach 2:
The patent implements a nested memory hierarchy where smaller, faster memory structures are contained within larger, slower ones. The L1 cache is nested within the L2 cache structure, which is nested within main memory, creating concentric layers of memory with varying access speeds and capacities.
2Productivity
If a multi-level memory hierarchy is implemented, then data access patterns are optimized, but device complexity increases
Solution Approach 1:
The patent implements dynamic memory management where the system can adaptively select which memory level to access based on data reuse patterns and access frequency. The memory hierarchy dynamically adjusts its behavior to optimize performance without requiring complex manual intervention.
Solution Approach 2:
The patent introduces intermediary buffer structures between different memory levels that automatically manage data transfer and caching. These intermediaries handle the complexity of coordinating multiple memory levels, allowing the system to access data efficiently without direct complex interactions between memory components.
3Loss of time
If data is stored and reused locally, then memory access latency is reduced, but the memory structure becomes more complex
Solution Approach 1:
The patent applies local quality optimization by creating localized cache structures (L1 cache) that are specifically optimized for frequently accessed data near the multiply accumulate layer. Different parts of the memory hierarchy have different qualities optimized for their specific access patterns, allowing local performance optimization without requiring the entire memory system to be complex.
Data Source
AI summary
A method of hierarchical structuring a multi-level memory in a convolutional neural network, includes partitioning a memory into a plurality of sections, partitioning the plurality of sections into a plurality of stripes, utilizing input data from the plurality of stripes in a MAC array, outputting an intermediate result from the MAC array to at least one of the plurality of stripes of a result buffer, looping back the intermediate result from the at least one of the plurality of stripes of the result buffer to at least one of the plurality of stripes of an input data buffer and outputting a final result from the at least one of the plurality of stripes of the result buffer to at least one of the plurality of stripes of an output buffer.