Neural Network Data Caching Circuit Using K-Row Feature Map Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The large number of layers in neural networks leads to significant amounts of input and intermediate data, limiting the speed and energy efficiency of neural network computations, as existing data caching methods are inefficient in managing this data.
Innovation Solution
A data caching circuit and method that utilizes K caching units to store every K consecutive rows of a feature map, simplifying cache addressing logic and reducing storage space by caching all data of the feature map, allowing for efficient data retrieval and processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If traditional data caching methods are used to store feature map data, then the cache memory size must be very large to accommodate all intermediate data, but this increases the device complexity and storage space requirements
Solution Approach 1:
The cache is divided into multiple caching units (first caching unit, second caching unit, etc.), each responsible for storing specific portions of feature map data. This segmentation allows the system to manage large amounts of data without requiring a single large complex cache structure, thereby reducing overall device complexity while maintaining sufficient storage capacity.
Solution Approach 2:
The patent transforms the traditional two-dimensional feature map storage into a three-dimensional caching structure by adding the time dimension through sequential storage of multiple feature maps. This dimensional transformation enables more efficient space utilization and reduces the complexity of cache addressing logic.
2Speed
If all intermediate data of the neural network is cached, then data retrieval speed can be improved, but the storage space requirements increase significantly
Solution Approach 1:
The patent extracts only the essential feature map data that is needed for subsequent convolution operations and stores it in the cache. By taking out only the necessary data elements rather than caching all possible intermediate data, the system achieves fast data retrieval while minimizing storage space requirements.
Solution Approach 2:
The cache stores multiple feature maps sequentially in a partial manner, keeping only the required number of feature maps available for the current computation stage. This partial caching approach provides sufficient data retrieval speed for neural network operations without allocating excessive storage space.
3Quantity of substance
If the cache stores every K consecutive rows of the feature map, then storage space is reduced, but the cache addressing logic becomes more complex
Solution Approach 1:
The patent merges multiple rows of the feature map (every K consecutive rows) into single caching units. This merging approach reduces the total number of cache locations needed, improving storage space efficiency. The addressing logic complexity is managed by systematically organizing which rows go to which caching units based on their positional relationships.
Data Source
AI summary
A data caching circuit and method are provided. The circuit is configured to cache data for a feature map calculated by a neural network, wherein a size of a convolution kernel of the neural network is K*K data, and a window corresponding to the convolution kernel slides at a step of S in the feature map, where K is a positive integer and S is a positive integer, the circuit comprising: a cache comprising K caching units, each caching unit being configured to respectively store a plurality of rows of the feature map, the plurality of rows comprising a corresponding row in every K consecutive rows of the feature map.