Reconfigurable NNA Memory Buffers for Conflict-Free Throughput
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
General-purpose processors face inefficiencies in executing neural networks due to memory access conflicts and high power consumption, as they lack specialized memory configurations for the distinct data requirements of different neural network types, leading to reduced throughput.
Innovation Solution
A neural network accelerator with reconfigurable memory that allocates specific buffers for activations, weights, and outputs, eliminating the need for dynamic arbitration by restricting memory locations to specific types of data, thereby optimizing memory usage based on the type of neural network being executed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a monolithic memory structure is used with no restrictions on memory unit usage, then memory flexibility and adaptability are improved, but memory access conflicts increase and throughput decreases
Solution Approach 1:
The memory is segmented into distinct buffer types (activation buffers, weight buffers, output buffers) with specific designated purposes. This segmentation eliminates access conflicts by ensuring that different clients access different memory regions, thereby maintaining throughput while preserving adaptability through configurable buffer sizes and numbers.
Solution Approach 2:
Different regions of the memory are assigned different qualities or functions based on their intended use. Activation buffers are optimized for storing intermediate computation results, weight buffers for storing network parameters, and output buffers for storing final results. This local differentiation resolves access conflicts while maintaining overall system flexibility.
2Adaptability or versatility
If a monolithic memory structure with unrestricted access is used, then memory adaptability is improved, but arbitration logic complexity and power consumption increase
Solution Approach 1:
By segmenting memory into dedicated buffers for different data types (activations, weights, outputs), the patent eliminates the need for complex arbitration logic. Each buffer type has its own designated access patterns, removing the need for dynamic conflict resolution mechanisms and reducing both complexity and power consumption.
Solution Approach 2:
Instead of allowing unrestricted memory access and using arbitration logic to manage conflicts, the patent inverts the approach by restricting access at the memory organization level. This inversion eliminates the need for complex arbitration mechanisms while maintaining memory adaptability through configurable buffer parameters.
3Adaptability or versatility
If general purpose processors are used for neural network computations, then processor versatility is improved, but computing efficiency and power consumption worsen
Solution Approach 1:
The patent applies local quality by creating specialized memory structures tailored to the specific needs of neural network computations. By organizing memory into activation buffers, weight buffers, and output buffers with optimized access patterns, the system achieves neural network-specific efficiency while maintaining the underlying general-purpose processor architecture.
Data Source
AI summary
Described herein is a neural network accelerator (NNA) with reconfigurable memory resources for forming a set of local memory buffers comprising at least one activation buffer, at least one weight buffer, and at least one output buffer. The NNA supports a plurality of predefined memory configurations that are optimized for maximizing throughput and reducing overall power consumption in different types of neural networks. The memory configurations differ with respect to at least one of a total amount of activation, weight, or output buffer memory, or a total number of activation, weight, or output buffers. Depending on which type of neural network is being executed and the memory behavior of the specific neural network, a memory configuration can be selected accordingly.